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| gO) ee VOnO ND AVS 





BA+ Blue Advantage Plus 

BHO Behavioral Health Management Organization 

CAHPS Consumer Assessment of Health Plans Survey 

CCP Community Care Plus 

CDC Centers for Disease Control and Prevention 

Chi-square A statistical test that is used to examine the probability of a change or difference 

in rates is due to chance. 

Cl Confidence Interval 

CMHC Community Mental Health Center 

CMS Centers for Medicare and Medicaid Services, U.S. Department of Health and 

Human Services 

CPT Current Procedural Terminology 

CY Calendar Year 

DHHS U.S. Department of Health and Human Services 

DHSS Missouri Department of Health and Senior Services 

DMS Division of Medical Services, Missouri Department of Social Services 

EPSDT Early, Periodic Screening, Diagnosis and Treatment 

EQR External Quality Review 

EQRO External Quality Review Organization 

FFS Fee-for-Service 

FG FirstGuard 

FHP Family Health Partners 

HCUSA HealthCare USA 

HCY Healthy Children and Youth, the Missouri Medicaid EPSDT program 

HEDIS Health Plan Employer Data and Information Set 

HIPAA Health Insurance Portability and Accountability Act 

HIS Health Information Systems 

HMO Health Maintenance Organization 
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ICD-9 


ICN 

ISCA 
LPHA 
LRA 

MBE 

MC+t 

MC+ MCOs 
MCO 

MDI 

MHP 
MMIS 
MOHSAIC 


NCPDP 
NCQA 
N.S. 


NSF/CMS 1500 


International Classification of Diseases, Ninth Revision, Clinical Modification, 
World Health Organization 


Individual Control Number 

Information Systems Capability Assessment 

Local Public Health Agency 

Lead Risk Assessment Guide, Healthy Children and Youth Program 
Minority-owned Business Enterprise 

The name of the Missouri Medicaid Program 

Managed Care Organizations for the Missouri Medicaid Program 
Managed Care Organization 

Missouri Department of Insurance 

Mercy Health Plan 

Medicaid Management Information System 


Missouri Health Strategic Architectures and Information Cooperative, Missouri 
Department of Health and Senior Services Public Health Immunization Registry 


National Council for Prescription Drug Program 
National Committee for Quality Assurance 


Not significant, indicating that a statistical test does not result in the ability to 
conclude that a real effect exists. 


National Standard Format/ Center for Medicare and Medicaid Services Form 
1500 


PCP Primary Care Physician 

PIHP Prepaid Inpatient Health Plan 

PIP Performance Improvement Project 

PRO Peer Review Organization 

QA & | Quality Assessment and Improvement Advisory Group 

QI/UM Quality Improvement/Utilization Management Coordinator. 

Coordinator 

SMA State Medicaid Agency, the Missouri Department of Social Services, Division of 
Medical Services 

SPHA State Public Health Agency, the Missouri Department of Health and Senior 
Services 

UB-92 Universal Billing Form 92 
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GLOSSARY AND OPERATIONAL DEFINITIONS 





Administrative Method 


Accuracy (Match) Rate 


Accuracy of a data field 


Accuracy of the State 
encounter claims 
database 
Commission (or 
surplus encounter 
claim) 

Completeness of a 


data field 


Confidence interval or 
level 


Encounter data 


The Administrative Method of calculating HEDIS Performance Measures 
requires the MCO to identify the denominator and numerator using 
transaction data or other administrative databases. The Administrative 
Method outlines the collection and calculation of a measure using only 
administrative data, including a description of the denominator (i.e., the 
entire eligible population), the numerator requirements (i.e., the indicated 


treatment or procedure) and any exclusions allowed for the measure. 


The ratio of identical or correct information in the medical record and 


the SMA relative to the number of encounters that took place. 
The extent to which an encounter claim field contains the correct type of 
information (e.g., numeric, alpha, alpha numeric) in the proper format 


(e.g., mm/dd/yyyy for date field). 


The extent to which encounters are being submitted for 100 percent of 


the services that are provided. | 


An encounter that is represented in the SMA encounter claims database 


but not the medical record; or a duplicate encounter. 


The extent to which an encounter claim field contains data (either 


present or absent). 


The range of accuracy of a population estimate obtained from a sample 


“Encounter data are records of health care services that have been 


provided to patients.” 2 


* Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 


Second Edition 
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Error An error in coding or recording an encounter claim. 


Fault (Error) Rate The ratio of missing and erroneous records relative to the total number 
of encounters that took place?. The rate at which the SMA encounter 
claims data does not match the medical record or the MCO paid 


encounter claims data (the converse of match rate). 


Hybrid Method Hybrid Method requires the MCO to identify the numerator through 
both administrative and medical record data. The MCO reports a rate 
based on members in the sample who are found through either 
administrative or medical record data to have received the service 


identified in the numerator. 


Interrater reliability A method of addressing the internal validity of a study by ensuring that 
(IRR) data are collected in a consistent manner across data collectors. 
Omission (or missing An encounter that occurred but is not represented in the State 
encounter claim) encounter claims database. 

Paid claim An encounter claim that has been paid by the MCO. 

Probability sample A sample in which every element in the sampling frame has a known, non- 


zero probability of being included in a sample. This produces unbiased 
estimates of population parameters that are linear functions of the 


observations from the sample data‘. 


Random sample Selection of sampling units from a sampling frame where each unit has an 


equal probability of selection. 


* Medstat (1999).: A Guide for States to Assist in the Collection and Analysis of Medicaid Manged Care Data. 
Medstat: Santa Barbara. Second Edition 

3 Centers for Medicare and Medicaid Services (2002). Validating Encounter Data: A protocol for use in 
conducting Medicaid External Quality Review activities, Final Protocol, Version 1.0, U.S. Department of Health 
and Human Services. 

*Levy, P.S., Lemeshow, S. (1999). Sampling of Populations: Methods and Applications, Third Edition. J ohn 
Wiley and Sons: New York. 
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Reasonableness of a The extent to which an encounter claim field represents a valid value 
data field (e.g., an actual procedure code, actual birth date); also referred to as 


validity of the data. 


Reliability The consistency of findings across time, situations, or raters. 
Sampling frame The population of potential sampling units that meet the criteria for 
selection (e.g., Medical encounter claim types from January |, 2004 


through March 31, 2004). 


Sampling unit Each unit in the sampling frame (e.g., an encounter). 
Simple sample Selection of sampling units from one sampling frame. 
Unpaid claim All unpaid and denied claims from the MCO; All claims not paid by the 


MCO either through capitation or through other payment methodology. 
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Introduction 


The United States Department of Health and Human Services (DHHS), Centers for Medicare and 
Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid 
Managed Care programs by an External Quality Review Organization (EQRO). External Quality 
Review is the analysis and evaluation by an approved EQRO of aggregate information on quality, 
timeliness, and access to health care services that MCOs and their contractors furnish to recipients 
of Medicaid managed care services. The Centers for Medicare and Medicaid Services (42 CFR §433 
and §438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations) 
rule specifies the requirements for evaluation of Medicaid managed care programs. The present 
report summarizes the findings of the first year of implementation of the mandatory activities for 
External Quality Review of the MC+ Managed Care Program in Missouri as conducted by Behavioral 
Health Concepts, Inc., a PRO-Like Entity certified by CMS to conduct External Quality Review 


(EQR) in all U.S. states and territories. 


The four mandatory protocols were implemented: |) Validating Performance Improvement 
Projects;> 2) Validating Performance Measures;® 3) Validating Encounter Data;” and 4) MCO 
Compliance with Managed Care Regulations. Two performance improvement projects (PIPs) at 
each MC+ MCO that were underway during the preceding |2 months were validated through a 
combination of self-selection and EQRO review, with final selection by the State Medicaid Agency 
(SMA; Missouri Department of Social Services, Division of Medical Services; DMS). The three 
performance measures validated were HEDIS 2004 measures of Adolescent Immunization Status, 
Combination #1, Adolescent Well-Care Visits, and the Use of Appropriate Medications for People 


with Asthma. Validation of encounter data examined the completeness, accuracy, and reliability of 


> Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Performance Improvement Projects: A protocol for use in Conducting Medicaid External Quality Review 
Activities, Final Protocol, Version 1.0, May 1, 2002. Washington, D.C.: Author. 

© Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Performance Measures: A protocol for use in Conducting Medicaid External Quality Review Activities, Final 
Protocol, Version 1.0, May 1, 2002. Washington, D.C.: Author. 

” Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Encounter Data: A protocol for use in Conducting Medicaid External Quality Review Activities, Final Protocol, 
Version 1.0, May 1, 2002. Washington, D.C.: Author. 

® Department of Health and Human Services. Centers for Medicare and Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR §400, 430, et al., Final 
Protocol, Version 1.0, February 11, 2003. Washington, D.C.: Author. 
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specific fields in the SMA database; and the extent to which paid claims in the SMA were 
represented in the medical records of MC+ Managed Care Members. The EQRO conducted all 
protocol activities, with the exception of the MCO Compliance with Managed Care Regulations 
Protocol. The SMA conducted these activities and requested the EQRO to review them 
(Compliance Review Analysis). Given that the mandatory protocols represented a major change in 
the scope and direction of the EQR, the EQRO invested substantial efforts in planning, advising, and 
preparing the SMA and MC+ MCOs to prepare for the rapid development and implementation 


cycle. 


Preparation for the 2004 External Quality Review 


PREPARATION WITH THE STATE MEDICAID AGENCY 

Effective July |, 2004 the State of Missouri contract for the External Quality Review of the MC+ 
Managed Care Program (State of Missouri Contract No: C301 154001, Amendment No.: 006) was 
revised to comply with federal requirements for states to contract with an external, independent 
entity to implement the mandatory protocols for External Quality Review. The Figure in Appendix 
| illustrates the SMA’s plan for the implementation of the EQR. Regularly scheduled monthly 
meetings with the SMA were canceled by the SMA in June 2004 due to the end of the previous 
contract cycle. The first monthly meeting for planning the scope of work, technical methods and 
objectives, and analyses was held by the SMA on September I9, 2004. Meetings were held with the 
SMA and the EQRO on September 14, 2004, September 24, 2004, October |5, 2004, October 22, 
2004, November 5, 2004, November 29, 2004, December |6, 2004, January 18, 2005, February 16, 
2005, March 9, 2005, April 14, 2005, and May I 1, 2005. Meetings scheduled for October |, 2004 
and October 5, 2004 were canceled by the SMA. The April 13, 2005 teleconference meeting was 
rescheduled by the EQRO in favor of a face-to-face meeting on April 14, 2005 to present and 
discuss preliminary findings. Additional meetings and teleconference calls were conducted as 


needed between SMA and EQRO personnel. 


At the first meeting in September 2004, a draft plan and questions for clarification were prepared, 
presented, and discussed by the EQRO for each of the mandatory protocols. During the month of 
September, the EQRO clarified the SMA’s objectives for each of the protocols, developed data 
requests for the SMA, prepared detailed proposals for the implementation and analysis of data for 
each protocol, and prepared materials for SMA review. Written proposals for each protocol were 


submitted on September 10, 2004 by the EQRO for SMA for review, discussion, revision, and 
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approval. By November 5, 2004, the EQRO had negotiated with the SMA the data request for State 
encounter data to be validated. Draft medical record review protocols were submitted to the SMA 
on September 24, 2004. All protocols were revised, finalized and submitted to the SMA by October 
2004. 


PREPARATION OF MC+ MCOs 

During October 2004, preparation of MC+ MCOs for the implementation of the 2004 EQRO was 
conducted by the EQRO Project Director and personnel. First, the EQRO Project Director 
contacted the primary contact person, the Quality Improvement Manager/Utilization Management 
Coordinator (QI/UM) or Medicaid Plan Administrator at each MC+ MCO to discuss the protocols, 
scope of work, data and information to be requested, and the estimated amount of time needed for 
MC+ MCOs to fulfill requests. This information was used to establish timelines for submission of 
requests for information and data to the EQRO. The EQRO Project Director and personnel 
conducted orientation to the protocols and discussions with MC+ MCQOs at the October 21, 2004 
meeting of the MC+ Managed Care All-Plan Meeting; and the October 28, 2004 meeting of the MC+ 
Managed Care Quality Improvement and Advisory (QA & |) Group. 


At the suggestion of the MC+ MCOs and the request of the SMA, 2-hour teleconference calls were 
arranged with each MC+ MCOs’ personnel and vendors to provide additional orientation to the 
protocols and the EQRO processes. The EQRO Project Director arranged the dates of the 
teleconference calls with MC+ MCO QI/UM Coordinators or Medicaid Plan Administrators. A 
detailed presentation, tentative list of data requests, and the proposals approved by the SMA were 
sent to MC+ MCOs prior to the teleconference orientation sessions. MC+ MCOs were requested 
to have all personnel involved in fulfilling the requests or in implementing activities related to the 
protocols (e.g., performance improvement projects to be validated, performance measures to be 
validated, encounter data requested) present at the teleconference calls. The orientation 
presentation is contained in Appendix 2. An SMA representative attended all conference calls and 
received minutes of the meetings taken by the EQRO upon completion of all the calls. Conference 
calls with EQRO and MC+ MCO personnel occurred between October 18, 2004 through October 
29, 2004. To avoid confusion and the inundation of many requests at once, the requests for 
information from MC+ MCOs were implemented in a staged approach from November 8, 2004 
through December 6, 2004. All communications (letters, general and specific instructions) were 


submitted for review, revision, and approval by the SMA prior to sending them to the MC+ MCOs. 
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DEVELOPMENT OF WORKSHEETS, TOOLS, AND RATING CRITERIA 

The EQRO Project Director, Research Associate, Assistant Project Director, a health services 
researcher, and a healthcare provider were responsible for the development of all worksheets and 
tools used for the EQRO. The EQRO Project Director revised the worksheet (Attachment B) of 
the Validating Performance Improvement Project Protocol to add detail for several items that were 
specific to the MC+ Managed Care Program. This was translated into a database by the Senior 


Research Associate and provided to the SMA. 


For the Validating Encounter Data Protocol, the EQRO Project Director developed the data analytic 
plan in collaboration with the SMA and revised methods and procedures based on the content, 
quality and format of data provided by the SMA and MC+ MCOs. The SMA selected the fields to 
validate for completeness, accuracy, and reliability of paid claims submitted by MC+ MCOs. The 
EQRO developed definitions of all field parameters for review, revision, and approval by the SMA. 
Encounter data critical field parameters were submitted to the SMA on January 14, 2005 for review 


and approval (received April 5, 2005). 


Input was obtained on the Validating Performance Measures Protocol worksheets by a Senior 
Research Analyst with experience calculating performance measures for hospitals, a health 
management information specialist, and a research assistant. The health information specialist and 


research assistant were also trained physicians. 


The SMA had already conducted the activities of the MC+ MCO Compliance with Managed Care 
Regulations Protocol through the state contract compliance monitoring process and the work of the 
EQRO involved the review and evaluation of this information (see Medicaid Program; External 
Quality Review of Medicaid Managed Care Organizations of 2003, CFR §438.58). The state contract 
for EQRO requires the EQRO to review the SMA’s activities with regard to the Protocol, however, 
additional policies and documents were requested prior to and during the on-site visits with MC+ 
MCOs when information was incomplete or unclear. To facilitate the review of compliance with 
federal regulations, the EQRO Project Director developed a cross-walk between the SMA contract 


requirements for Medicaid managed care and the federal Medicaid Managed Care Regulations. 
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An expert in MC+ Medicaid Managed Care Program policy and programming reviewed and refined 
the tool in collaboration with the MC+ Managed Care Program consultant who has participated in 
the EQRO for the past three years. Feedback on inconsistencies between the Medicaid Managed 
Care contract and federal requirements was provided immediately to the SMA. The SMA did not 
have a rating system for the Protocol, so one was developed by the EQRO for SMA review and 
finalization of ratings based on EQRO findings. A rating system was proposed by the EQRO and 
approved by the SMA in January 2005. The SMA requested the EQRO to provide ratings, to be 
reviewed and finalized by the SMA. The SMA was provided with the tool for review and approval 
on January 21, 2005. The SMA provided state compliance review information to be reviewed by the 
EQRO for all MC+ MCOs by February 25, 2005. The EQRO staff and the consultant reviewed all 
available materials and met with SMA staff for two full days on February 8 and 9, 2005 to clarify SMA 
comments and compliance ratings; and identify issues for follow-up at site visits. Final ratings were 


provided by the SMA on April 19, 2005. 


The following sections summarize the aggregate findings and conclusions for each of the mandatory 
protocols. The full report is organized according to each protocol and contains detailed 
descriptions of the technical methods, objectives, findings, and conclusions (strengths, areas for 
improvement, and recommendations). The full report provides MCO to MCO comparisons and 


individual MC+ MCO summaries for each protocol. 


Validation of Performance Improvement Projects 


For the Validating Performance Improvement Projects (PIP) Protocol, there were two PIPs that 
were underway during the previous |2 month period validated at each MC+ MCO, for a total of 14 
PIPs validated. Eligible PIPs for validation were identified by the MC+ MCOs, SMA, and the EQRO. 
PIPs are to be aimed at studying the effectiveness of clinical or non-clinical interventions that 
improve processes highly associated with healthcare outcomes, and/or healthcare outcomes 
themselves. They are to be carried out over multiple re-measurement periods to measure 
improvement, the need for continued improvement, or stability in improvement as a result of an 
intervention. Under the State contract for Medicaid Managed Care, MC+ MCOs are required to 


have two active PIPs. This was the first year of implementation of the PIP Protocol, and specific 
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feedback and technical assistance was provided to each MC+ MCO by the EQRO during the site 


visits for improving study methods, data collection, and analysis. MC+ MCOs have been moving 


from a quality improvement and monitoring approach to one revolving around the evaluation of 


quality improvement activities and interventions. 


STRENGTHS 


Across MC+ MCOs, there were strengths noted in the identification of processes and outcomes 
in need of improvement, the calculation of measures of healthcare processes and outcomes, and 
in the interventions designed to improve the outcomes of care for MC+ Members. 


Four of the 14 PIPs were rated as credible and valid approaches to determining the effectiveness 
of interventions. 


One PIP was identified as a Best Practice for the Improving Care for Asthmatics, as the results 
showed demonstrable, statistically significant improvement in healthcare outcomes for MC+ 
Members. 


Three of the four credible PIPs were underway, but were designed with the potential to 
produce credible and valid findings for the identification of additional Best Practice approaches 
to delivering healthcare to MC+ Members. 


MC+ MCOs possess the clinical expertise for designing sound interventions, identifying 
measures for evaluating the interventions, and identifying barriers to implementing interventions. 


MC+ MCOs were responsive to and engaged in the process of EQRO technical assistance for 
PIP development and expressed interest in additional technical assistance. 


AREAS FOR IMPROVEMENT 


Areas for improvement included the need for the development of logic models on which to base 
study design, data collection, and analyses for evaluating the effectiveness of interventions as well 
as Statistical significance testing for supporting claims of effectiveness. It is critical that there is a 
logical link between the study questions, measures of implementation, measures of outcomes, 
and analysis of findings to address directly the problem(s) or issue(s) identified. 


PIPs should be ongoing, active projects evaluating aspects of the quality improvement program 
at each MC+ MCO and should be monitored on at least a quarterly basis for identification of 
barriers to implementation of the intervention and data collection. Several MC+ MCOs 
appeared to develop PIPs in response to the EQRO validation process rather than having active 
projects in place. 
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RECOMMENDATIONS 


|. The implementation of valid, credible PIPs will require additional training, consultation, and 
technical assistance for MC+ MCOs in the development and planning of PIPs, and in data 
analysis. It is recommended that the SMA support MC+ MCO efforts in obtaining consistent 
technical assistance from a qualified healthcare services evaluator. The EQRO provided 
technical assistance to the QA & | Advisory Group and on-site during the MC+ MCO site visits; 
and provided references for logic model development and healthcare evaluation methods to the 
MC+ Managed Care Program. Additional hands-on workshops for PIP development are 
recommended. 


2. The MC+ MCQOs have used various forms and formats for submitting PIPs for SMA and EQRO 
review. There is no specific format required or preferred by the EQRO or SMA at the present 
time, and it is the EQROs recommendation that MC+ MCOs use the format that best meets 
individual MCO needs. The National Committee for Quality Assurance (NCQA) Quality 
Improvement Activity (QIA) Form provides good structure for the calculation and reporting of 
measures, but does not include areas for all aspects of PIPs. Some MC+ MCOs effectively used 
this format, but need to modify it to include all PIP elements (e.g., study question, specific 
sources of data, interpretation of study findings, and identification of barriers to 
implementation). 


3. The implementation of a statewide PIP is scheduled for 2007. It is recommended that study 
topic development and planning for implementation begin one year prior to implementation, to 
identify issues in data submission, aggregation, and analysis across MC+ MCOs. It is 
recommended that the meta-analytic study or multi-site evaluation design be implemented and 
coordinated by an external entity with no relationship or investment in any of the MC+ MCOs 
and with experience in conducting such evaluations. 


4. Suggested topics for statewide PIPs that appear to apply to all MC+ MCOs include the 
management of members with asthma, the identification and treatment of lead poisoning, and 
the identification of high risk pregnancies. Of the PIPs validated, three addressed the treatment 
of asthma and four addressed blood lead level screening or treatment. Given that these studies 
have been underway for at least twelve months and MC+ MCOs have begun to collect data, 
these will serve as a foundation for the planning of a statewide PIP. The Best Practice for 
Improving Care for Asthmatics may also be a viable and successful approach to a statewide PIP. 
This has a basis and rationale for widespread implementation. It is recommended that these 
topics be considered by the SMA and the MC+ QA & | Advisory Group. 


Validation of Performance Measures 


The Validating Performance Measures Protocol requires the validation or calculation of three 
performance measures at each MC+ MCO by the EQRO. The measures selected for validation by 
the SMA were measures that are required to be submitted by MC+ MCOs annually, and by the 
State Public Health Agency (SPHA; Missouri Department of Health and Senior Services; DHSS) for 
all Health Maintenance Organizations (HMOs) operating in the State of Missouri. They were: I) 
HEDIS 2004 Adolescent Immunization Status, Combination #1; 2) HEDIS 2004 Adolescent Well- 
Care Visits; and 3) HEDIS 2004 Use of Appropriate Medications for People with Asthma. There are 


detailed specifications for the calculation of these measures that were developed by the NCQA, a 
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national accrediting organization for managed care organizations. The EQRO examined the 


information systems, detailed algorithms, MC+ MCO extract files, medical records, and data 


submissions provided to the SPHA to conduct the validation activities of this protocol. The data 


reported to the SPHA are based on MC+ MCO performance during 2003. MC+ MCOs were given 


an opportunity to review and correct the data presented to the SPHA; and were provided with the 


opportunity to review EQRO findings for comment and correction. 


STRENGTHS 


MC+ MCOs have strong management information systems for the documentation and payment 
of services to providers and the tracking of member information. These systems and the 
processes for performance measure calculation are well documented and MC+ MCOs retain 
qualified personnel for the programming of data specifications to calculate performance 
measures. 


Five of the seven MC+ MCOs incorporated external data (State Public Health Immunization 
Registry; the Missouri Health Strategic Architecture and Information Cooperative; MOHSAIC) 
to capture immunizations delivered outside the MC+ MCO for the HEDIS 2004 Adolescent 
Immunization Status, Combination #| measure. 


All MC+ MCOs produced the HEDIS 2004 Use of Appropriate Medications for People with 
Asthma Measure in a manner that was Substantially Compliant with the specifications. 


Four of the seven MC+ MCOs had significantly higher rates than the average for all MC+ MCOs 
on the HEDIS 2004 Use of Appropriate Medications for People with Asthma measure. The 
rates were higher than the National Commercial and Medicaid rates for this measure. 


One of the two MC+ MCOs that were Fully Compliant with the calculation of the HEDIS 2004 
Adolescent Immunization Status, Combination # | measure had a rate higher than the National 
Commercial and Medicaid rates for this measure. 


Four of the seven MC+ MCOs employed NCQA-certified software for the rate calculation 
process, which requires passing test file procedures by NCQA for accurate and valid 
programming and calculation of the specifications. 


In response to the EQRO exit interview comments and preliminary findings for this Protocol, 
several MC+ MCOs developed corrective action plans for improving the process of 
documentation, rate calculation, oversight, and staff training for the calculation of performance 
measures. 


AREAS FOR IMPROVEMENT 


The HEDIS 2004 Adolescent Immunization Status Combination #| measure was not able to be 
validated for five of the seven MC+ MCOs, and thus is not a valid measure of the effectiveness 
of the MC+ Managed Care Program for this service to adolescents. 


The HEDIS 2004 Adolescent Well-Care Visits Measure was not able to be validated for three of 
the seven MC+ MCOs, and thus is not a valid measure of the effectiveness of the MC+ Managed 
Care Program for this service. 
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There was substantial error in the calculation of the Adolescent Immunization Status, 
Combination #! and the Adolescent Well-Care Visits measures. Some error was related to the 
medical record submission rate, while some was due to the lack of compliance with measure 
methodology and specifications. If all of the medical records had been submitted, it is possible 
that these measures would have been validated for more MC+ MCOs. The Adolescent 
Immunization Status, Combination #I| measure is a complex measure to calculate, requiring 
expertise in the specifications for the measure as well as the programming of multiple events 
and integration of multiple data sets. 


In many cases, there was limited organizational knowledge of the process of calculating 
measures. This was related to personnel turnover and the use of vendors for various aspects of 
claims administration and the rate calculation process. 


There was limited ability to provide data extract files in a standard format for validation. MC+ 
MCOs had difficulty providing the numerator and denominator extract files for validation, 
making it difficult to identify and validate eligible members and conduct sampling for the 
validation process. Attempts were made to provide orientation, specifications, and technical 
assistance for numerator and denominator extract files for EQRO validation. There were 
limitations in the medical record validation process as a result of difficulty identifying hybrid 
numerator hits in the MC+ MCO extract files. 


One MC+ MCO that did not use NCQA-certified software was found to have used the 
incorrect measurement year for the Adolescent Well-Care Visits measure, indicating the utility 
of this software. 


There was variability in the implementation of the Hybrid Method, and few MC+ MCOs 
provided medical record review tools and/or manuals to assess whether medical record reviews 
are conducted in a similar manner across MC+ MCOs. The EQR validation used the same 
criteria for numerator events across all MC+ MCOs in the validation process. In addition, not 
all MC+ MCOs reviewed all of the medical records sampled for the Hybrid Method of 
calculation. 


RECOMMENDATIONS 


It is recommended that the SMA use measures of MC+ Managed Care Program performance 
only after the validity of the measure can be established for all MC+ MCOs. This would include 
the HEDIS 2004 Adolescent Immunization Status, Combination #| and Adolescent Well-Care 
Visits measures based on this evaluation. The validity of the Adolescent Immunization Status, 
Combination #| measure should be improved through the incorporation of the State Public 
Health Immunization Registry (Missouri Health Strategic Architecture and Information 
Cooperative; MOHSAIC) data, proper implementation of the Hybrid Method, and reliable 
medical record review procedures. 


lt is recommended that the SMA discontinue the validation of performance measures that are 
Substantially or Fully Compliant with State standards using the Validation of Performance 
Measures Protocol. For measures the SMA wishes to use as measures of program performance, 
continue to validate them until all MC+ MCOs come into Full or Substantial Compliance for the 
calculation of the measures. The Adolescent Immunization Status and Adolescent Well-Care 
Visits measures should be revalidated for the HEDIS 2006 reporting year or later, as the 
measures have already been calculated for the HEDIS 2005 reporting year by MC+ MCOs 
without the benefit of the present validation and recommendations. 
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3. For other measures requiring retrospective data more than one year prior to the measurement 
year (e.g., the Adolescent Immunization Status measure), the SMA should require the Hybrid 
Method of validation and the use of all possible external data sources to ensure completeness 
and validity of the data used for the measurement of performance. 


4. \f the SMA validates HEDIS measures in the future, the SMA should revise the Information 
System Capability Assessment (ISCA) to reduce duplication with the HEDIS Baseline 
Assessment Tool (BAT). The EQRO provided to the SMA a written summary of areas of 
overlap between the two tools based on the BAT. Future revisions should be focused on the 
purpose of the encounter data validation and performance measure validation processes to be 
used by the EQRO and the SMA. 


5. MC+MCOQOs should develop internal expertise in the calculation of performance measures, the 
submission of required performance measures for Missouri, and for the MC+ Managed Care 
Program. 


6. MC+MCOQOs should document policies, procedures, processes, and responsibilities of personnel 
and vendors involved in calculating and reporting HEDIS measures for the MC+ Managed Care 
Program. This should include vendor oversight and review of files and data produced to 
calculate the measures. 


7. Whenever possible, MC+ MCOs should use NCQA-certified software or vendors who use 
NCQA-certified software to calculate measures and identify errors in programming and 
extracting data for the calculation of performance measures. 


8. Performance measure calculation for MC+ MCOs should be conducted with the benefit of 
external reporting, such as is done through the reporting process of HEDIS measures to the 
SPHA for HEDIS measures. For example, Mental Health Indicators should be compiled by an 
entity external to the MC+ MCOs with the requisite technical expertise to ensure consistent, 
valid collection of data across MC+ MCOs for comparison and program evaluation purposes. 


9. Duplication across Mental Health Indicators and HEDIS measures should be reduced as much as 
possible to avoid confusion about what is being measured. 


10. The SMA should consider validating the HEDIS measures of 7-day and 30-day Follow-up After 
Psychiatric Hospitalization. Given the complexity of multiple event measures, the SMA should 
also consider validating MC+ MCO calculation of the HEDIS Childhood Immunization Status 
measure. 


11. Performance measures reported across MC+ MCOs should be conducted by a qualified entity 
external to the MC+ MCOs. 


Encounter Data Validation 


Encounter claims data are used by SMAs to conduct rate setting and quality improvement 
evaluation. Before SMA encounter claims data can be used, it is necessary to establish the extent to 
which the data for critical fields (e.g., diagnosis and procedure codes, units and dates of service, 
member and provider identifiers) are complete (there is information in each field), accurate (the 
information is of the right size and type), and valid (the data represent actual dates or procedure and 


diagnosis codes). Several critical fields for each of six claim types (Medical, Dental, Home Health, 
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Inpatient, Outpatient, Hospital, and Pharmacy) were identified by the SMA and examined by the 
EQRO for completeness, accuracy, and validity using an extract file from SMA paid encounter 
claims. To examine the extent to which the SMA encounter claims database was complete (the 
SMA encounter claims database represents all claims paid by MC+ MCOs), the level and consistency 
of services was evaluated by examining the rate of each of six claim types for each MC+ MCO. 
Another method of comparing the representativeness (or completeness) of the SMA encounter 
claims database is to compare data in the SMA encounter claims database to the medical records of 
members. This was examined for a random sample of medical records for each MC+ MCO, 
comparing the diagnosis and procedure codes in the SMA encounter claims database with 
documentation in MC+ Member medical records. The completeness of the SMA paid encounter 
claims was to be compared with MC+ MCO records of paid and unpaid claims. This analysis was 


unable to be conducted due to MC+ MCO data submission issues. 


STRENGTHS 


|. The majority of critical fields evaluated for each of the six encounter claim types were accurate, 
complete, and valid. 


2. The Pharmacy, Dental, and Home Health critical fields contained valid data for the analysis of 
paid encounter claims. 


3. Of the medical records received for review, there was a match rate of 70.9% for procedures 
and 73.8% for diagnoses with the data in the SMA encounter claims extract file. 


AREAS FOR IMPROVEMENT 


|. The examination of the completeness, accuracy and validity of SMA encounter claims data was 
not able to assess the level of representativeness of the SMA encounter claims database relative 
the claims paid by MC+ MCOs. 


2. The EQRO was able to validate 53.3% of procedures and 55.4% of diagnoses from the medical 
records. This was related to the rate of submission of medical records for review and the 
inability to validate records not received. This is also lower the expected rate of 70%. 


3. Medical records that did not have diagnosis or procedure codes that matched those in the SMA 
encounter claims database were in error primarily due to missing or illegible data. 


4. MC+ MCOs were not able to produce paid and unpaid claims in valid formats for encounter 
data validation to assess the completeness of paid claims as represented in the SMA encounter 
claims database or to identify omission and commission errors related to encounter data 
submission. 


5. Comparison data from other SMAs with similar programs and encounter claim types and who 
have validated their data is needed to evaluate the completeness of the MC+ Managed Care 
Program encounter claims database. 


Performance Management Solutions Group | 


a Division of Behavioral Health Concepts, Inc. 


2716 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 
www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


6. The volume and consistency of services across MC+ MCOs and claim types were highly variable, 
with no patterns across MC+ Managed Care Regions observed. 


RECOMMENDATIONS 


|. The SMA Dental, Home Health, and Pharmacy encounter claim types should be compared with 
MC+ MCO paid and unpaid encounters once a format for MC+ MCO submission is identified. 


2. MC+MCOs should explore potential reasons for variation in claim types and in the proportion 
of each claim type to all claim types. 


3. It is recommended that the SMA add edits for invalid encounter claims data for specific fields in 
the Medical, Inpatient, and Outpatient Hospital claim types. 


4. \t is recommended that MC+ MCOs emphasize to providers the importance of documentation 
of diagnostic and procedural information in the medical record. 


5. It is recommended that the SMA and MC+ MCOs identify valid and consistent file layouts for 
paid and unpaid encounter claim extract files to be validated. 


6. It is recommended that the SMA identify comparative data from other states regarding 
encounter claim rates for each claim type. 


7. Additional methods for assessing the completeness of the SMA encounter claims database for 
the MC+ Managed Care Program should be employed, such as examining the dates of service 
and submission of each claim type on a monthly basis over the course of one year. 


MC+ MCO Compliance with Managed Care Regulations 


The purpose of the protocol to monitor MCO Compliance with Managed Care Regulations was to 
provide an independent review of MC+ MCO activities and assess the quality outcomes of 
timeliness and access to the services provided by the MC+ MCOs. The protocol used two main 
sources of information to determine compliance with the federal regulations. They were document 
review and interviews with MC+ MCO personnel. This combination of information was designed to 


provide the SMA with a better understanding of organizational performance at each MC+ MCO. 


The policy and practice in the operation of each MC+ MCO was evaluated against the seventy (/0) 
regulations related to operating a Medicaid managed care program. The regulations were grouped 
into three main categories: Enrollee Rights and Protections, Quality Assessment and Improvement, 
and Grievance Systems. The category of Quality Assessment and Improvement was subdivided into 
three subcategories: Access Standards, Structure and Operation Standards, and Measurement and 
Improvement. Initially, the SMA reviewed each MC+ MCOs’ policy to determine compliance with 


the requirements of the MC+ Medicaid Managed Care Contract. These determinations and their 
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application to the requirements of the federal regulations were assessed. Additional document 


review occurred when the MC+ MCO policy submission did not meet MC+ Medicaid Managed Care 


contract requirements, or where clarification was necessary. A set of interview questions specific 


to each MC+ MCO was developed to elicit information that validated organizational practice and 


that explored issues not fully addressed in the documents. 


STRENGTHS 


Four MC+ MCOs Met or Partially Met all applicable federal regulations and related State 
compliance requirements for Medicaid managed care. 


MC+ MCOs demonstrated a strength in compliance with federal regulations for grievance and 
appeals processes and procedures. This is likely due to concerted SMA efforts to ensure MC+ 
MCO compliance with regulations. 


Across MC+ MCOQOs, an investment in the development of programs was observed that often 
exceeded the strict requirements of the MC+ Medicaid Managed Care contract. The MCt+ 
MCOs used their resources to ensure that MC+ Members had access to quality healthcare 
services. 


MC+ MCOs incorporated methods of quality assessment and program improvement into their 
daily operations. 


Across MC+ MCOs, there was adherence to the policy and practice for operating grievance and 
appeals systems. Six of the seven MC+ MCOs exhibited substantial compliance with all required 
policies. All MC+ MCOs described how they used data gathered from the grievance and 
appeals process to generate program improvements. 


MC+ MCOs were responsive and cooperative with the EQRO process by submitting available 
materials and data despite personnel changes, limited resources, and first year EQRO 
implementation planning and issues. 


AREAS FOR IMPROVEMENT 


A consistent area for improvement for the MC+ MCOs was monitoring, development, and 
timely submission of policies to ensure compliance with the MC+ Medicaid Managed Care 
contract and with the federal Medicaid Managed Care Regulations. The quality of practice 
observed at several of the MC+ MCOs was overshadowed by their inability to produce required 
policies or complete the submission and approval process. 


The MC+ MCOQs all voiced concerns about problems they encountered. Two prominent 
examples were the difficulties encountered in enrolling specific specialties into their network, 
and their challenge to maintain accurate member contact information. The MC+ MCOs all 
provided examples that exhibited depth in the area of creative problem solving. It would appear 
that with an effort to combine this resourcefulness they could find a common resolution to 
some of their shared problems. 


All the MC+ MCOs maintain health information systems that can provide a wealth of data 
regarding their MC+ Members. There was not consistent utilization of this information to assist 
in development of quality initiatives that could enhance program development. 
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During the process of the EQRO, there were a number of HIPAA violations in the submission 
of encounter data (sending aspects of the limited data set via email and fax when specifically 
requested by the EQRO not to do so) and lack of security of medical records (Protected Health 
Information). 


RECOMMENDATIONS 


It is recommended that MC+ MCOs prioritize submission of documents used for member 
communication of rights such as the Member Handbook, Marketing and Educational Materials, 
and dissemination plans. 


lt is recommended that the SMA update contract language to be consistent with federal 
regulations [438.1 0(f)(5)]. Specific feedback was provided to the SMA by the EQRO upon 
discovery of areas of incongruence. 


MC+ MCOs should address concerns regarding access to specialty services for dental, 
behavioral health, orthopedic, neurological, and other specialties known to be in short supply. 
The use of utilization data, monitoring indicators, and surveys is recommended to examine and 
evaluate on a quarterly basis the needs of MC+ Members with Special Health Care Needs. 


The SMA should develop a compliance protocol tool and rating system for the Compliance 
Protocol. A model was developed and proposed by the EQRO and approved by the SMA for 
use in the present evaluation. The SMA should revise this tool and develop criteria for rating 
each item based on the findings of the present year. This should be supplied to and reviewed 
with the EQRO prior to the initiation of validation of compliance activities. 


MC+ MCOs should conduct routine review and revision of internal policies and procedures 
annually or when contract revisions occur, and submit revisions to the SMA for review and 
approval. 


MC+ MCOs should implement HIPAA training for policies and procedures at orientation for 
new staff and at least annually for all staff. 


It is recommended that the SMA make available to the MC+ MCOs the tool completed by the 
EQRO for this Protocol to facilitate program compliance. 


To facilitate timely and accurate submissions of MC+MCO information to the EQRO, it is 
recommended that the SMA provide to the EQRO all final protocols, worksheets, rating 
systems and criteria to be used, data, and content areas to be reviewed no later than November 
30 in the EQRO cycle. 
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SECTION ONE: VALIDATION OF PERFORMANCE IMPROVEMENT 





PROJECTS 


Definition 


A Performance Improvement Project (PIP) is defined by the Centers for Medicare and Medicaid 
Services (CMS) as “a project designed to assess and improve processes, and outcomes of care.... 
that is designed, conducted and reported in a methodologically sound manner.” The Validating 
Performance Improvement Projects Protocol specifies that the EQRO conduct three activities in the 
validation of two PIPs at each MCO that have been initiated, are underway, were completed during 
the reporting year, or some combination of these three stages. The State Medicaid Agency (SMA; 
the Department of Social Services, Division of Medical Services; DMS) elected to examine projects 
that were underway during the preceding |2 months. Criteria for identification as a PIP that are 
outlined in the CMS protocols include the following: 


PIPs need to have a pre-test, intervention, and post-test 

PIPs need to control for extraneous factors 

PIPs need to include an entire population 

Pilot projects do not constitute a PIP 

Satisfaction studies alone do not constitute a PIP 

Focused studies are not PIPS; A focused study is designed to assess processes and outcomes on one-time 
basis, while a PIP is to improve processes and outcomes of care over time. 


The State of Missouri contract for Medicaid Managed Care (RFP NO. B3Z03182, Contract 
Amendment 002, 08/25/2003) describes the following requirements for MC+ MCQOs in conducting 
PIPs: 


Performance Improvement Projects: The health plan must conduct performance improvement projects that are 

designed to achieve, through ongoing measurements and intervention, significant improvement, sustained over 

time, in clinical care and nonclinical care areas that are expected to have a favorable effect on health outcomes 

and member satisfaction. The health plan must report the status and results of each project to the state 

agency as requested. The performance improvement projects must involve the following: 

e Measurement of performance using objective quality indicators. 

Implementation of system interventions to achieve improvement in quality. 

Evaluation of the effectiveness of the interventions. 

Planning and initiation of activities for increasing or sustaining improvement. 

Completion of the performance improvement project in a reasonable time period so as to generally allow 

information on the success of performance improvement projects in the aggregate to produce new 

information on quality of care every year. 

e Performance measures and topics for performance improvement projects specified by CMS in consultation 
with the state agency and other stakeholders. 
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Purpose and Objectives 


The purpose and objectives of the present review were to evaluate the soundness and results of 
PIPs implemented by MC+ MCOs; to ensure that MC+ MCOs carry out at least two PIPs each year, 
with examination of the stability or variability in change over multiple years; and to determine the 


best approach to a statewide project in the future. 


Technical Methods 


There are three evaluation activities specified in the protocol for Validating Performance 


Improvement Projects. Activity One consists of ten steps: 


Activity One: Assessing the MCOs /PIHP’s Methodology for 
Conducting the PIP 


|. Step One: Review the selected study topic(s) 

Step Two: Review the study question(s) 

Step Three: Review selected study indicator(s) 

Step Four: Review the identified study population 

Step Five: Review sampling methods (if sampling was used) 
Step Six: Review the MCOs/PIHP’s data collection procedures 
Step Seven: Assess the MCOs/PIHP’s improvement strategies 


Step Eight: Review data analysis and interpretation of study results 


2. 
oy 
4. 
oy 
6. 
7. 
8. 
9. 


|”’ 


Step Nine: Assess the likelihood that reported improvement is “rea 
improvement 
. Step Ten: Assess whether the MCO/PIHP has sustained its documented 


improvement 





Activity Two (Verifying PIP Study Findings) is optional, and involves auditing PIP data. Activity Three 
(Evaluate Overall Reliability and Validity of Study Findings) involves the judgment of whether the 
results and conclusions drawn from the PIP are valid and reliable. Activities One and Three were 


conducted by the EQRO. 
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TIME FRAME AND SELECTION 

Two projects that were underway during the preceding |2 months at each MC+ MCO were 
selected for validation. The projects to be validated were reviewed with MC+ MCO QI/UM 
Coordinators, SMA staff, and other MC+ MCO staff throughout the month of October 2004. The 
intent was to identify projects which were mature enough for validation (i.e., planned and in the 
initial stages of implementation), yet still underway during calendar year 2004. During the process 
of submission, one MC+ MCO requested and was granted the request to change one PIP submitted 


for validation. 


PREPARATION OF MC+ MCOs 

All MC+ MCOs were contacted directly by the EQRO Project Director between September [5, 
2004 and October | 3, 2004 to discuss the implementation of the 2004 EQR. Technical assistance 
was provided to MC+ MCOs at the quarterly MC+ Managed Care Program Quality Assessment and 
Improvement Advisory (QA & |) Advisory Group meeting sponsored by the SMA on October 28, 
2004. Additional technical assistance and orientation to the Validating Performance Improvement 
Projects Protocol was provided at the quarterly MC+ Managed Care Program QA & | Advisory 
Group meeting on January 19, 2005 and at the quarterly MC+ Managed Care Program All-Plan 
Meeting on October 21, 2004 (see Appendix 3). The January presentation included the definition of 
a PIP, elements of the protocol, and a sample PIP incorporating all elements of the Protocol. 
Further, MC+ MCOs were provided with individual orientation to the Validating Performance 
Improvement Projects Protocol and the EQR implementation through individual two-hour 
teleconference calls with each MC+ MCO and personnel involved in the implementation and 
conduct of PIPs. These occurred between October 18, 2004 and October 29, 2004. Letters and 
specific instructions were sent to MC+ MCOs on November 8, 2004 for PIP documentation to be 


submitted by November 23, 2004 (see Appendix 2). 


Procedures for Data Collection 


The evaluation involved review by the Senior Research Associate and EQRO Project Director of all 
materials submitted by MC+ MCOs including, but not limited to the materials listed below. During 


the orientation teleconference calls, MC+ MCOs were strongly encouraged to review Attachment B 
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to the Validating Performance Improvement Projects Protocol and ensure they included information, 
documents, tools, and any other information necessary for evaluation based on this tool. 


e Narrative descriptions 

e Problem identification 

e Hypotheses 

e Evaluation Questions 

e Description of Intervention(s) 

e Methods of sampling, measurement 

e Planned Analyses 

e Sample tools, measures, surveys, etc. 

e Baseline data source and data 

e Cover letter with clarifying information 

e Overall analysis of the validity, reliability of each study 
e Scaled scoring system if desired by the State 

e Evaluate the results of the PIPs, not just the methods 


The EQRO Project Director met with personnel responsible for planning, conducting, and 
interpreting the findings of PIPs during site visits conducted between March I, 2005 and March 23, 
2005. This review focused on findings of projects submitted at the end of 2004. MC+ MCOs were 
requested to be prepared to review databases and any data collection forms not submitted with the 
original request. One hour was scheduled during the on-site visits to conduct follow-up questions, 
review databases and provide technical assistance to MC+ MCOs regarding the planning, 
implementation, and credibility of findings from PIPs. In addition to individual clarifying questions 
regarding each PIP, the following questions were used to gather more information regarding the PIPs 
during the on-site interviews: 


e Who was the Project Leader? 

e How was the topic identified? 

e How was the study question determined? 

e What were the findings? 

e What was the intervention? 

e What was the time period of the study? 

e Was the intervention effective? 

e How do you know? 

e = Why or why not? 

e What does the MCO want to study or learn from their PIPs? 


All PIPs were evaluated by a Senior Research Associate with three years of experience conducting 
the EQR in Missouri and with six years of data analysis and research experience. In addition, 
projects were reviewed and followed-up on-site by the EQRO Project Director, with final ratings 
based on all information obtained from MC+ MCO submissions and site visits. Minimum 
qualifications for PIP reviewers are described in the training materials developed by the EQRO for 
the evaluation of MC+ Managed Care Program PIPs. Below is an excerpt from the Instructions to 


Reviewers developed by the EQRO. 
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Reviewers must be trained by the EQRO Project Director prior to conducting reviews. The EQRO Project 
Director will review and finalize all comments, ratings, and recommendations. As part of this process, qualified 
reviewers must develop familiarity with the CMS Protocol for Validating Performance Improvement Projects, the 
Objectives and Technical Methods for the Validation of Performance Improvement Projects (PIPs), and the 
operational definitions provided in this Protocol. Qualified reviewers must have a minimum of a Bachelor’s 
degree in a health or human services related field, with three to five years of professional experience conducting 
healthcare, policy, or program evaluation and research. 


ANALYSIS 

All PIPS submitted by MC+ MCOs prior to the site visits were reviewed using an expanded version 
of the checklist for conducting Activities One, Steps | through 10; and Activity Three (Judgment of 
the Validity and Reliability of the PIPs) of the Validating Performance Improvement Projects Protocol 
Attachment B (see Appendix 3). Because specific criteria may not have been applicable for projects 
that were underway at the time of review, a specific item may not have applied in a particular case. 
Criteria were rated as “Met” if the item was applicable to the PIP, if there was documentation 
addressing the item, and if the item could be deemed Met based on the study design. The 
proportion of items rated as “Met” was compared to the total number of items that were applicable 
for the particular PIP. Given that some PIPs were underway in the first year of implementation, it 
was not possible to judge the interpretation of study results, validity of improvement, or sustained 
improvement (Steps 8-10). The final evaluation of the validity and reliability of studies underway 
were based on the potential for the studies to produce credible findings. Detailed 
recommendations and suggestions for improvement were made for each item as deemed 
appropriate, and are presented in the individual MC+ MCO summaries. Although an item may have 
been met, suggestions and recommendations may have been provided as technical assistance for 
future improvement. The following are the general definitions developed for evaluating each 


criteria. 


Met: Credible, reliable, and valid methods for the item were documented. 


Partially Met : Credible, reliable, or valid methods were implied or able to be established for part of the 
item. 


Not Met: The study did not provide enough documentation to determine whether credible, reliable, 
methods were employed; errors in logic were noted; or contradictory information was 
presented or interpreted erroneously. 


Not Applicable: Only to be used in Step 5, when there is clear indication that the entire population was 
included in the study and no sampling was conducted; or in Steps 8 through 10 when the 
study period was underway for the first year. 
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Findings 


Below are the PIPs identified for validation at each MC+ MCO. 


Community Care Plus Emergency Room Utilization 
Twenty-Four Hour Provider Access Monitor 

Mercy Health Plan Improving Quality of Life of Members Living with Asthma 
High Risk Pregnancy 

HealthCare USA Member Reminder Initiative 
School-Based Program Participation 2004 

Missouri Care Increasing Appropriate Childhood Immunization Practices 


Increasing Blood Lead Testing by PCPs 


Family Health Partners Emergency Room Utilization 
Lead 
FirstGuard Identification of MC+ Children with Elevated Blood Lead Level 


Improving Asthma Medication Management 
Blue Advantage Plus Lead Testing Improvement Project 


Improving Care for Asthmatics 


STEP 1: SELECTED STUDY TOPICS 

Study topics should be selected through data collection and analysis of comprehensive aspects of 
member needs, care, and services; address a broad spectrum of key aspects of member care and 
services; and include all enrolled populations without excluding certain members. Four of the |4 
PIPs addressed blood lead level toxicity; three addressed care for members with asthma; two 
addressed preventive care (participation in school-based program and immunizations); two 
addressed non-clinical areas (provider access monitoring and member reminders); two addressed 
emergency room utilization; and one addressed high risk pregnancies. Tables | and 2 show the 
ratings for each item and PIP by MC+ MCO. Six of the |4 PIPs (42.9%) provided data demonstrating 
the extent of the need and rationale for the PIP, discussed literature supporting the activities to be 
undertaken, and provided benchmark comparison data (see Table 2, Step I.1). MC+ MCOs 
addressed a broad spectrum of key aspects of member care and services (9 of the 14 PIPs, 64.3%, 
Met this criteria and 3 Partially Met this criteria; Step |.2). Clinical and nonclinical interventions 


were identified and aspects of enrollee care and services that were related to the identified problem 
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were described. One study was focused on emergency room utilization alone, which is considered 
too narrow a scope for a PIP. Utilization or cost issues may be examined through a PIP, but should 
not be the sole focus of the study. There were few descriptions of the member populations 
targeted for intervention in the PIPs. Because the MC+ MCOs vary widely in the member 
populations they serve (e.g., other state Medicaid managed care members, commercial members, or 
Medicare members), it was not possible to determine the extent to which the PIP identified, 
addressed, and measured the needs of the MC+ Managed Care Program population. In addition, 
PIPs should specifically indicate whether all enrolled populations within the MC+ Managed Care 
Program were included in the interventions. Finally, age and demographic characteristics should be 
described. One (Increased Blood Level Testing by PCPs) of the 14 PIPs (7.1%) Met this criteria 
(Step |.3). 


STEP 2: STUDY QUESTIONS 

Study questions are statements in the form of a question that describe the potential relation 
between the intervention, the intended outcome, and the direction of the relation. They should be 
specific enough to suggest the study methods and the outcome measures. Although specific 
statements were rarely provided in the form of a question, it was possible to identify the intended 
question and purpose of the study for six of the 14 (42.9%) PIPs (Step 2.1). For some, the study 
purpose identified was not consistent with the remainder of the PIP (the target population, 


interventions, measures, or methods). 
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Table |. Performance Improvement Project Validation Findings by MC+ MCO. 
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Source: BHC, Inc., 2004 External Quality Review Performance Improvement Project Validation. 
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STEP 3: STUDY INDICATORS 

All PIPs either Met or Partially Met the criteria for defining and describing the calculation of study 
indicators. Half (50%) of the PIPs Met the criteria for using objective, clearly defined, measurable 
indicators while half Partially Met the criteria (Step 3.1). The calculation of measures was described 
and explained. In some cases, when well-known measures were used (e.g., Health Employer Data 
Information Set; HEDIS; Consumer Assessment of Health Plans Survey; CAHPS), there was little or 
no description of the methods (e.g., Administrative or Hybrid Method) and formulas for calculating 
the measures. Again, because MC+ MCOs vary in their method of calculation, details regarding the 
measures and methods of calculating them should be included in PIPs. Four of the [4 PIPs (28.6%) 
identified at least one study indicator that was related to health or functional status; or to processes 
of care strongly associated with outcomes (Step 3.2). The link between the intervention and the 


outcomes measured by the PIP should be made explicit in the narrative. 


STEP 4: STUDY POPULATIONS 

None of the PIPs Met the criteria for clearly defining all the MC+ Managed Care Program Members 
to whom the study question(s) and indicator(s) were relevant (Step 4.2). Ten of the 13 PIPs (71.4%) 
Met the criteria for defining the study populations. When applicable, it should be clearly stated that 
no specific populations (e.g., | 115 Waiver, 1915b Waiver, children in state custody, children served 
by the consent decree, children with special health care needs) were excluded from the PIP. 
Otherwise, the selection criteria should clearly describe the MC+ Managed Care Member 
populations included in the PIP and their demographic characteristics. Two of the I3 PIPs (15.4%) 
described data collection approaches indicating that data for all members to whom the study 
question applied were collected (Step 4.2). There was some misunderstanding of sampling (e.g., 
‘The sample size was determined by how many people we were able to contact during the quarter 
being measured’) and the difference between a sample and a population (“‘all eligible members were 
sampled’). Most of the time, there were vague or ambiguous descriptions that made it difficult to 
determine how data were collected and how participants were identified. For example, it was not 
clear whether “all high volume PCPs” included family practitioners, OB/GYNs, internists, and/or 
specialists chosen as PCPs. There were some exclusions of members that appeared to be related to 
measure specification (e.g., only HEDIS-eligible members). Although measures of convenience 


(those calculated already for other purposes) are acceptable and encouraged, the interventions 
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themselves should not exclude members based on the length of eligibility or service status. The 
limitation of measures (e.g., applicable to only eligible members) should be described in the PIP 


methods. 


STEP 5: SAMPLING METHODS 

Four PIPs actually employed sampling techniques, while others used the term to refer to an entire 
population studied. None of the four PIPs for which sampling was conducted described the 
population, sampling techniques, rationale, or confidence intervals and margins of error to be used 
in measurement. The type of sample (e.g., convenience, random) sampling methods (e.g., simple, 


cluster, stratified) should be described. 


STEP 6: DATA COLLECTION PROCEDURES 

Six of the 13 PIPs (46.2%) described the data to be collected with adequate detail and description of 
the units of measurement used (Step 6.1). Seven of the 14 (50.0%) PIPs clearly specified the sources 
of data (e.g., claims, members, providers, medical records) for each measure (Step 6.2). Some MC+ 
MCOs used the National Committee for Quality Assurance (NCQA) Quality Improvement Activity 
(QIA) Form to write PIPs, which provides a structure for reporting measures and data sources. 
However, when there is more than one source of data, it is important that the MC+ MCO 
specifically state the sources of data for each measure. Two of the seven PIPs (14.3%) clearly 
described systematic and reliable methods of data collection (Step 6.3). In most cases, the data 
collection procedures were not described. It was not possible to judge the reliability or credibility 
of a PIP without sufficient detail regarding data collection processes, procedures, or frequency. Of 
the eight PIPs that used some source of survey or data collection tool, none were able to fully 
ensure that consistent, accurate data were able to be collected over time (Step 6.4). However, five 
Partially Met this criteria. Few MC+ MCOs provided instruments for review. One PIP indicated 
that “the database” provided for accurate data collection. When using surveys, medical records, or 
telephone protocols for data collection, it is important to provide the tool for review, discuss the 
piloting of the tool, and discuss training and interrater reliability for the recording of information on 
the tool. Standard provider and consumer surveys provide manuals describing the characteristics of 
instruments that should be incorporated into the narrative of the PIP. Two PIPs (14.3%) included a 
complete data analysis plan, while one Partially Met the criteria for specifying a plan (Step 6.5). This 
should be developed prior to the implementation of the PIP based on the study questions, expected 


relation between the intervention(s) and outcome(s) being measured (i.e. independent and 
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dependent variables), the method(s) of data collection, and the nature of the data (e.g., nominal, 
ordinal, scale). One of the 14 (7.1%) PIPs identified in the narrative the personnel and qualifications 
of personnel involved in the design, implementation, data analysis, and interpretation of the PIP (Step 
6.6). Although MC+ MCO staffs interviewed on-site were knowledgeable about the PIPs and 


methods, the PIP evaluation team qualifications and roles were rarely described in PIP materials. 


STEP 7: IMPROVEMENT STRATEGIES 
Six of the 14 (42.9%) of the PIPs identified reasonable interventions to address the barriers identified 
through data analysis and quality improvement processes undertaken. The nature of identification of 


the barriers, a description of barriers, and a plan for addressing barriers should be described. 


STEP 8: DATA ANALYSIS AND INTERPRETATION OF STUDY RESULTS 

None of the || PIPs that had conducted analyses did so according to the data analysis plan (Step 
8.1). Only two PIPs described any data analysis plan against which to determine if the plan was 
implemented. Of the || PIPs that presented baseline or re-measurement data, three (27.3%) 
presented numerical findings accurately and clearly (Step 8.2). In some instances, data were 
presented in formats different from those described in the calculation of measures (e.g., presenting 
percents in graphic format while the description of the calculation of measures indicated rates per 
|,000). Five PIPS Partially Met this criteria. Axis labels and units of measurement should be 
reported in Tables and in Figure legends. The data year for benchmark data should also be labeled. 
Of the eleven PIPs that presented at least one re-measurement period, one (9.1%) indicated the re- 
measurement period for all of the measures identified in the study (Step 8.3). Of the ten PIPs 
describing the findings, one (Improving Care for Asthmatics; | 1.1%) described the extent to which 


the intervention was effective (Step 8.4). 


STEP 9: VALIDITY OF IMPROVEMENT 

Five of the ten PIPs (50.0%) with re-measurement points used the same method at re-measurement 
as the baseline measurement (Step 9.1). One PIP described the use of different measures from 
baseline than re-measurement. As discussed with MC+ MCOs during site visits, this is acceptable as 
long as the rationale and limitations are described. If possible, the baseline measure should be 
recalculated consistent with the re-measurement method to ensure validity of reported 
improvement and comparability of measurement over time. The same source of measures should 


also be used at re-measurement points. Two PIPs employed statistical significance testing to 
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document quantitative improvements in care (Improving Asthma Medication Management and 
Improving Care for Asthmatics; Step 9.2). One showed statistically significant improvement over 
multiple re-measurement points and the other showed stability on the first re-measurement. One 
of nine (I 1.1%) PIPs reporting improvements had face validity, meaning that the reported 
improvement is judged to have been related to the intervention applied (Step 9.3). There was little 
to no discussion or interpretation of findings by MC+ MCOs. After reporting findings, there should 
be some interpretation as to whether the intervention or some other factors may have accounted 
for improvement, decline, or lack of change (Step 9.4). Then, barriers should be identified and 


addressed for the next cycle of the PIP, or reasons for discontinuing the PIP should be described. 


STEP 10: SUSTAINED IMPROVEMENT 

Of the five PIPs examining multiple measurement points over time, only two (40.0%) PIPs used 
statistical significance testing to demonstrate improvement. One PIP showed statistically significant 
improvement over several measurement points. Across all MC+ MCOs, the range in proportion of 
criteria that were Met for each PIP validated was 0.0% through 62.5%. Across all PIPs validated 
statewide, 25.5% of criteria were met. All sources of available data were used to develop the ratings 
for the PIP items. However, EQRO comments were developed based on the written 
documentation and presentation of findings. For at least three MC+ MCOs, it appeared that PIP 
materials and narratives were developed in a short time frame in response to the EQRO request for 
information. In other cases, there was not enough information provided to validate the PIPs, but 
on-site interviews revealed in-depth knowledge of the PIPs. PIPs are to be ongoing, with periodic 
re-measurement points. At least quarterly re-measurement is recommended to provide timely 
feedback to the MC+ MCO regarding the need to address barriers to implementation. MC+ MCO 
personnel involved in PIPs had extensive experience in clinical service delivery, quality improvement, 
and monitoring activities, but not the requisite experience in designing valid evaluation studies using 
sound data collection and analysis methods. This requires technical expertise in health services 
research and/or program evaluation design. The development of a logic model describing the 
interventions, short-, intermediate-, and long-term goals and measures, and an analytic plan at the 
outset of a project are key aspects of implementing a project that will reliably identify the impact of 


an intervention. 
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Table 2. Summary of Performance Improvement Project Validation Ratings by Item, All MC+ MCOs. 
All MC+ MCOs 
ee 
Partially Met; Not Met pV eye) tCer-] eo) (=) Rate Met 
1.1 4 13 46.2% 
14 64.3% 
14 7.1% 
14 42.9% 
50.0% 
28.6% 
0.0% 
15.4% 
0.0% 
33.3% 
0.0% 
46.2% 
50.0% 
14.3% 
0.0% 
14.3% 
7.1% 
42.9% 
0.0% 
27.3% 
9.1% 
10.0% 
50.0% 
11.1% 
11.1% 
11.1% 
Step 10: Sustained Improvement 20.0% 


Number Met | 4} gg} 22} 295] 25.1% 


Note: Percent Met = Number Met/ Number Applicable; Item refers to the Protocol specifications. 
Source: BHC, Inc., 2004 External Quality Review Performance Improvement Project Validation. 
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Table 3 shows a summary of the final audit rating assessing the level of confidence in the credibility 
or potential credibility of PIP findings based on the current status of the projects. Two of the |4 
PIPs produced highly credible findings partly as a result of conducting statistical significance testing 
and demonstrating the significance of change or stability over re-measurement points. The 
proportion of items met for each PIP may not correspond with the final ratings. This is due to the 
fact that although discrete criteria may have been met (e.g., specifying the calculation of measures), 
the overall study design may not have appropriately incorporated the measures or the measures 
may not have been valid indices of the intervention(s). The final audit ratings were based on the 


entire project design and implementation. 
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Table 3. Validity and Reliability of Performance Improvement Project Results. 

PIP Name 

Member Reminder Initiative Low Confidence 
School-Based Program Participation 2004 Low Confidence 
Increasing Appropriate Childhood Immunization Practices Low Confidence 
Increasing Blood Lead Testing by PCPs Low Confidence 
Emergency Room Utilization Low Confidence 

Lead Low Confidence 
Improving Quality of life of Members Living with Asthma Moderate Confidence 
High Risk Pregnancy Moderate Confidence 


Emergency Room Utilization Not Credible 


Twenty-Four Hour Provider Access Monitor Not Credible 


Identification of MC+ Children with Elevated Blood Lead Level Low Confidence 
Improving Asthma Medication Management Low Confidence 


Lead Testing I! mprovement Project High Confidence 





Improving Care for Asthmatics High Confidence 
Note: Not Credible = There is little evidence that the study will or did produce results that could be attributed to the 
intervention(s); Low Confidence = Few aspects of the PIP were described or performed in a manner that would produce 
some confidence that findings could be attributed to the intervention(s); Moderate Confidence = Many aspects of the PIP 
were described or performed in a manner that would produce some confidence that findings could be attributed to the 
intervention(s); High Confidence = The PIP study was conducted or planned in a methodologically sound manner, with 
internal and external validity, standard measurement, and data collection practices, and appropriate analyses to calculate 
that there is a high level of confidence that improvements were a result of the intervention. A 95% to 99% level of 
confidence in the findings was or may be able to be demonstrated. 
Source: BHC, Inc., 2004 External Quality Review Performance Improvement Project Validation. 


STATEWIDE PERFORMANCE IMPROVEMENT PROJECT 

A second objective of the EQRO with regard to PIPs was to obtain input from MC+ MCOs on the 
implementation of a statewide PIP. There has been some discussion in the MC+ Quality Assessment 
and Improvement (QA & I) Advisory Group about the SMAs desire to implement a statewide PIP 
for MC+ Members. At the time of the EQRO, there were plans to implement a project in 2007, but 


no specific topic or plan for implementation had been identified. 
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lt may be appropriate to identify a statewide PIP topic based on the topics chosen by MC+ MCOs 
for PIPs. All three of the Western Region MC+ MCOs had lead poisoning identification or 
treatment as PIPs underway during the previous |2 months, but it was reported that lead poisoning 
was less of a concern in the Eastern Region, with maternity issues presenting a greater concern. 
There were three MC+ MCOs across the Eastern and Western MC+ Regions that identified the 
treatment of asthma for PIPs. One of these PIPs was identified as a Best Practice and the other 


demonstrated moderate potential for producing credible findings. 


Conclusions 


Based on the PIP validation process, at least four MC+ MCOs (Mercy Health Plan, Family Health 
Partners, First Guard and Blue Advantage Plus) had active and ongoing PIPs as part of their quality 
improvement program. The following summarizes the strengths, areas for improvement, and 
recommendations based on the findings of the Validation of Performance Improvement Projects 


activity. 


STRENGTHS 


|. There was good topic identification and good intervention development for PIPs. MC+ MCO 
personnel were well qualified to identify study topics, areas in need of improvement, and 
interventions to address barriers to quality of care and health outcomes. 


2. MC+MCOs implemented interventions aimed at key aspects of enrollee care and services, such 
as medication management, risk identification and stratification for various levels of care, 
monitoring provider access, blood lead level screening, and preventive care. 


3. The descriptions of studies and intended purposes were often clear and relevant to the rationale 
and problem identified. 


Study indicators were well-defined and methods for calculating them were detailed. 
Study designs specified the data to be collected and the sources of data for each measure. 
Quality improvement strategies were appropriate and relevant. 


The same methods and measures over re-measurement periods were used. 


eS oY eM US 


The Improving Care for Asthmatics PIP was underway for several re-measurement periods and 
identified statistically significant improvement in healthcare outcomes over re-measurement 
periods. This PIP meets criteria as a Best Practice, and the intervention should be shared with 
other MC+ MCOs to assist with the improvement of healthcare outcomes for all MC+ 
Members with asthma. 


9. Additional PIPS (Improving the Quality of Life of Members Living with Asthma, Lead Testing 
Improvement Project, and High Risk Pregnancy) were judged to be likely or highly likely to 
produce credible and valid findings to identify Best Practices. 


Performance Management Solutions Group 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouR!I MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


. MC+ MCO personnel were attentive and responsive to technical assistance for the 


implementation of PIPs and presented goals or plans for improving the process in the future. 


. MC+ MCOS instituted personnel and/or role changes to improve the focus and expertise of 


personnel responsible for PIP implementation. 


AREAS FOR IMPROVEMENT 


Study questions, although implied, were not stated in many of the PIPs. The study question 
frames the purpose, intended outcomes, goals, and methods for the study. 


There were two PIPS that were focused almost exclusively on cost and/or utilization, and one 
that appeared to be focused on improving documentation of care alone. PIPs should directly 
address the outcomes of care on health or functional status. 


Data analysis plans were not described in most of the PIPs. The method of data collection and 
compilation is often dependent on the plan for data analysis. A plan should be developed prior 
to conducting a PIP, and modified as needed to be able to test for significant in improvement or 
stability of performance on the outcome measures over time. Data analyses could not be 
evaluated without a description of the plan. 


There was limited technical expertise at MC+ MCOs for designing methodologically sound 
evaluation studies, conducting sampling and statistical analysis, and interpreting findings. 


There were PIPs underway or ongoing that resulted in the potential for credible findings. 


Although it appears that many MC+ MCOs conduct PIPs on an ongoing basis as part of their 
quality improvement program, at least three MC+ MCOs appeared to have developed or 
conducted PIPs in response to the request by the EQRO for materials. 


RECOMMENDATIONS 


It is recommended that MC+ MCOs obtain additional training, assistance and expertise for the 
design, statistical analysis, and interpretation of PIP findings. 


In the design of PIPs, MC+ MCOs need to use generally accepted practices for program 
evaluation to conduct PIPs. In addition to training on the development of PIPs and on-site 
technical assistance, references to the CMS protocol, “Calculating Performance Measures” were 
provided by the EQRO to each MC+ MCO along with references on logic model development, 
health services evaluation methods, and health statistics (see Appendix 3). 


PIPs should be conducted on an ongoing basis, with at least quarterly measurement of some 
indices to provide data about the need for changes in implementation, data collection, or 
interventions. 


It is recommended that a statewide PIP be identified by the SMA and the MC+ QA & | Group 
for planning and implementation one year prior to the planned implementation. The topic 
should be chosen based on the present study findings and current topic identification of MC+ 
MCOs as important processes that are likely to affect a broad segment of the population and 
key healthcare outcomes. Asthma treatment and management appears to be an important topic 
with interventions that have been shown to be effective with MC+ Members. 
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SECTION TWO: VALIDATION OF PERFORMANCE MEASURES 





Definition 


The Validating Performance Measures Protocol requires the EQRO to validate three performance 
measures at each MC+ MCO, as selected by the State Medicaid Agency (SMA; the Missouri 
Department of Social Services, Division of Medical Services; DMS). The three performance 
measures validated by the EQRO were the HEDIS 2004 measures of Adolescent Immunization 
Status, Combination #1; Adolescent Well-Care Visits; and the Use of Appropriate Medications for 
People with Asthma (Combined Rate). Protocol activities involved the review of the data 
management processes of the MC+ MCO, evaluation of algorithmic compliance with performance 
measure specifications, and verification of either the entire set or a sample of the performance 


measures to confirm that the reported results are based on accurate service information. 


Purpose and Objectives 


The objectives for validating performance measures were to evaluate the accuracy of Medicaid 
performance measures reported by, or on behalf of, MC+ MCOs; and determine the extent to 
which MC+ MCO-specific performance measures calculated by MC+ MCOs (or by entities acting on 
behalf of MC+ MCOs) followed specifications established by the SMA and the State Public Health 
Agency (SPHA; Missouri Department of Health and Senior Services; DHSS) for the calculation of the 


performance measure(s). 


Technical Methods 


The reliable and valid calculation of performance measures is necessary for calculating statewide 
rates, comparing MC+ MCO performance with other MC+ MCOs, and for comparing State and 
MC+ MCO performance with national benchmark data for Medicaid managed care and/or 
Commercial Managed Care Organization (MCO) members to evaluate program effectiveness and 
access to care. State of Missouri requirements for MC+ MCO performance measurement and 
reporting were reviewed. The Missouri Code of State Regulations (19 CSR §10-5.010 Monitoring 
Health Maintenance Organizations) contains provisions requiring all Health Maintenance 
Organizations (HMOs) operating in the State of Missouri to submit to the State SPHA member 


satisfaction survey findings and quality indicator data in formats conforming to the National 
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Committee for Quality Assurance (NCQA) Health Employer Data Information Set (HEDIS) Data 
Submission Tool (DST) and all other HEDIS Technical Specifications? for performance measure 
descriptions and calculations. Additionally, the State of Missouri contract for Medicaid Managed 
Care (B3Z03182; Revised Attachment 6, Quality Improvement Strategy, 08/13/2003) stipulates that 
MC+ MCOs will follow instructions of the SPHA for submission of HEDIS measures. The three 
measures selected by the SMA for validation were required to be calculated and reported by MC+ 
MCOs to both the SMA and the SPHA for MC+ Managed Care Members. The HEDIS 2004 
Technical Specifications were reviewed for each of the three measures and are summarized below 


(see Tables 4, 5, and 6). 


HEDIS 2004 ADOLESCENT IMMUNIZATION STATUS, COMBINATION #1 

The following is the general definition of the Adolescent Immunization Status measure and the 
specific parameters for the Combination #| measure, an Effectiveness of Care measure as identified 
by NCQA. 


The percentage of enrolled adolescents who turned | 3 years old during the measurement year, who were 
continuously enrolled for |2 months immediately prior to their | 3th birthday and who had a second dose of 
MMR, three hepatitis B and one VZV by the member’s | 3th birthday. The measure also calculates two 
separate combination rates. Combination #1 consists of adolescents who received the second MMR and three 
hepatitis B vaccinations as specified above. 


Table 4. HEDIS 2004 Technical Specifications for Adolescent Immunization Status, Combination #1. 


Product lines: Medicaid and commercial (report each product line separately). 


Age: Adolescents who turn 13 years old during the measurement year (2003). 


Continuous enrollment: Twelve months prior to the member’s 13th birthday. 


No more than one gap in enrollment of up to 45 days during the 12 months prior 
to the 13th birthday. To determine continuous enrollment for a Medicaid 

Allowable gap: beneficiary for whom enrollment is verified monthly, the member may not have 
more than a 1-month gap in coverage (i.e., a member whose coverage lapses 
for 2 months [60 days] is not continuously enrolled). 


Anchor date: Enrolled on the member’s 13th birthday. 
Benefit: Medical. 
Event/ diagnosis: None. 


Denominator: The eligible population. 


MMR CPT: 90707, 90710; ICD-9:99.48 
Numerators: Measles CPT:CPT: 90705, 90708; ICD-9:055*, 99.45 
Administrative Method: Mumps CPT:90704, 90709; ICD-9:072*, 99.46 
Rubella CPT:90706, 90708, 90709; |CD-9:056*, 99.47 





? National Committee for Quality Assurance (2003). HEDIS 2004, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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Table 4. HEDIS 2004 Technical Specifications for Adolescent Immunization Status, Combination #1 - 
Continued. 


Hybrid Method: Hepatitis B CPT:90723, 90740, 90743**, 90744, 90746, 90747, 90748; ICD- 
9:V02.61*, 070.2*, 070.3* 


For immunization information obtained from the medical record, the MCO may 
count members where there is evidence that the antigen was rendered from: 
a note indicating the name of the specific antigen and the date of the 
immunization, or a certificate of immunization prepared by an authorized health 
care provider or agency including the specific dates and types of immunizations 
administered 
A second dose of MMR by the member’s 13th birthday. To be compliant, a 
Measles, Mumps, Rubella member must have received either: 
(MMR): one MMR on or between the member’s 4th and 13th birthdays, or 
two MMRs on or between the member’s 1st and 4th birthdays. 
Three hepatitis B vaccinations with different dates of service by the member’s 
13th birthday. The MCO may count a member compliant if the member received 
Hepatitis B (Hep B): the complete two-dose hepatitis B regimen identified by CPT code 90743. 
Members are also compliant if they receive one dose of the two-dose regimen 
(90743) and two other doses of hepatitis B. 
Adolescents who received the second MMR and three hepatitis B vaccinations as 
specified above. 

Exclusion (optional): Adolescents who had a contraindication for a specific vaccine may be excluded 
from the denominator for all antigen rates and the combination rates. The 
denominator for all rates must be the same. The MCO that chooses to exclude 
contraindicated adolescents may do so only for those adolescents where the 
administrative data does not indicate that the contraindicated immunization was 
rendered. The exclusion must have occurred by the member’s 13th birthday. 
The MCO should look for contraindications as far back as possible in the 
member’s history and may use the contraindications and codes in Table E2-B 
(see HEDIS Technical Specifications) to identify allowable exclusions. 

Source: National Committee for Quality Assurance (2003). HEDIS 2004, Volume 2: Technical Specifications. Washington, D.C.: 
NCQA. 


Combination #1: 





HEDIS 2004 ADOLESCENT WELL-CARE VISITS 
The following is the definition of the Adolescent Well-Care Visits measure, an Access to Care 
measure, as defined by the NCQA. 


The percentage of enrolled members who were | 2—21 years of age and continuously enrolled during the 
measurement year and who had at least one comprehensive well-care visit with a primary care practitioner or 
an OB/GYN practitioner during the measurement year. 


Table 5. HEDIS 2004 Technical Specifications for Adolescent Well-Care Visits. 
Product lines: Medicaid and commercial (report each product line separately). 


Ages: 12-21 as of December 31 of the measurement year (2003). 


Continuous The measurement year (2003). 

enrollment: 

Allowable gap: Members who have had no more than one gap in enrollment of up to 45 days during 
the measurement year. To determine continuous enrollment for a Medicaid member 


for who enrollment is verified monthly the member may not have more than a 1- 
month gap in coverage (i.e., a member whose coverage lapses for 2 months [60 days] 
is not considered continuously enrolled). 

Anchor date: Enrolled as of December 31 of the measurement year. 


Benefit: Medical. 


Event/ diagnosis: None. 
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Table 5. HEDIS 2004 Technical Specifications for Adolescent Well-Care Visits - Continued. 


Denominator: A systematic sample drawn from the MCO’s eligible population. The MCO may reduce 
its sample size using the current year’s administrative rate or the prior year’s audited, 
product-line-specific rate. 

Numerators: At least one comprehensive well-care visit with a primary care practitioner or an 
OB/GYN practitioner during the measurement year, as documented through either 
administrative data or medical record review. The primary care practitioner does not 
have to be the practitioner assigned to the member. The MCO may count services 
that occur over multiple visits toward this measure as long as all services occur within 
the time frame established in the measure. 

Administrative CPT Codes: 99383-99385, 99393- 99395 

Method: |CD-9-CM Codes: V20.2, V70.0, V70.3, V70.5, V70.6, V70.8, V70.9 

Hybrid Method: Documentation in the medical record must include a note indicating a visit to a 
primary care practitioner or OB/GYN practitioner, the date on which the well-care visit 
occurred, and evidence of all of the following: 

a health and developmental history (physical and mental) 

a physical exam health education/anticipatory guidance. 
Source: National Committee for Quality Assurance (2004). HEDIS 2004, Volume 2: Technical Specifications. Washington, D.C.: 
NCQA. 





HEDIS 2004 USE OF APPROPRIATE MEDICATIONS FOR PEOPLE WITH ASTHMA 
The following is the definition of the Use of Appropriate Medications for People with Asthma 


measure, an Effectiveness of Care measure as defined by NCQA. 


The percentage of enrolled members 5—56 years of age during the measurement year who were identified as 
having persistent asthma during the year prior to the measurement year and who were appropriately 
prescribed medication during the measurement year. 


Table 6. HEDIS 2004 Technical Specifications for Use of Appropriate Medications for People with Asthma. 
Product lines: Medicaid and commercial (report each product line separately). 


Ages: 5-56 by December 31 of the measurement year (2003). For each product line, the 
measure should be reported for each of three age stratifications (based on age as of 
December 31 of the measurement year; 2003) and as a combined rate: 

5-9 year-olds 

10-17 year-olds 

18-56 year-olds 

combined rate. 

The combined rate is the sum of the three numerators divided by the sum of the three 
denominators. 

Continuous The measurement year and the year prior to the measurement year (2003). 

enrollment: 

Allowable gap: No more than one gap in enrollment of up to 45 days during each year of continuous 
enrollment. To determine continuous enrollment for a Medicaid beneficiary for whom 
enrollment is verified monthly, the member may not have more than a 1-month gap in 
coverage during each year of continuous enrollment year. 

Anchor date: Enrolled as of December 31 of the measurement year (2003). 


Benefits: Medical. Pharmacy during the measurement year (2003). 


Event/ diagnosis: Persistent asthma. To identify all members with persistent asthma, the MCO should 
use all applicable coding schemes to count members that meet the criteria listed 
below: 

Acute inpatient: CPT Codes: 99221-99223, 99231-99233, 99238, 99239, 99251- 
99255, 99261-99263, 99291, 99292; UB-92 Revenue Codes: 10X-16X, 20X-22X, 987 
ED services: CPT Codes: 99281-99285, 99288; UB-92 Revenue Codes: 45X, 981 
Outpatient visit : CPT Codes: 99201-99205, 99211-99215, 99217-99220, 99241- 
99245, 99271-99275; UB-92 Revenue Codes: 456, 510, 515-517, 520, 521, 523, 526, 
76X, 770, 779, 982, 983, 988 
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Table 6. HEDIS 2004 Technical Specifications for Use of Appropriate Medications for People with Asthma 
— Continued. 


Identify members as having persistent asthma who, during the year prior to the 
measurement year (2002), had any of the following: 
at least one Emergency Department (ED) visit based on the visit codes below with 
asthma (I1CD-9 code 493) as the principal diagnosis 
at least one acute inpatient discharge based on the visit codes below, with asthma as 
the principal diagnosis 
at least four outpatient asthma visits based on the visit codes in Table E14-A, with 
asthma as one of the listed diagnoses and at least two asthma medication dispensing 
events* 
at least four asthma medication dispensing events* (ij.e., an asthma medication was 
dispensed on four occasions). 
For a member identified as having persistent asthma because of at least four asthma 
medication dispensing events, and leukotriene modifiers were the sole asthma 
medication dispensed, the member must: meet any one of the other four criteria, or 
have at least one diagnosis of asthma in any setting in the year prior to the 
measurement year. 
Exclusion: The MCO may exclude from the eligible population all members diagnosed with 

emphysema and chronic obstructive pulmonary disease (COPD) any time on or prior to 
December 31 of the measurement year, as identified by the following codes: 
Emphysema ICD-9 Codes: 492, 506.4, 518.1, 518.2; 
COPD ICD-9 Codes: 491.2, 493.2-493.22, 496 491.20, 491.21, 492.0, 492.8, 496, 
518.1, 518.2, 506.4 

Source: National Committee for Quality Assurance (2004). HEDIS 2004, Volume 2: Technical Specifications. Washington, D.C.: 

NCQA. 





METHODS OF CALCULATING PERFORMANCE MEASURES 

According to HEDIS technical specifications, there are two methods of calculating performance 
measures: I) the Administrative Method and 2) the Hybrid Method. The Adolescent Immunization 
Status, Combination #1 and the Adolescent Well-Care Visits measures permit the MCO to calculate 
the percentages (also referred to as rates) using either the Administrative Method or the Hybrid 
Method. The Use of Appropriate Medications for People with Asthma measure is required to be 


calculated using the Administrative Method. 


The Administrative Method involves examining claims and other databases (administrative data) to 
calculate the number of members in the entire eligible population who received a particular service 
(e.g., immunizations, well-care visits, or medication). The eligible population is defined by the HEDIS 
technical specifications. Those cases in which administrative data show that the member received 
the service(s) examined are considered “hits”, or “administrative hits.” The HEDIS technical 


specifications provide acceptable administrative codes for identifying an administrative hit. 


The Hybrid Method entails the selection of a random sample of members from the eligible 
population during the measurement year. For the Hybrid Method, administrative data are examined 


to select members eligible for the measure and to identify the number of members who received 
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the service(s) as evidenced by a claim submission or through external sources of administrative data 
(e.g., State Public Health Agency Vital Statistics or Immunization Registry databases). Those cases in 
which there are no administrative data indicating that the member received the service or all of the 
services required to be an “administrative hit” are identified for medical record review. 
Documentation of all or some of the services in the medical record in combination with 


administrative data is considered a “hybrid hit.” 


Administrative hits and hybrid hits are summed to form the numerator of the rate of members 
receiving the service of interest (e.g., immunizations, well-care visit, and medication). The 
denominator of the rate is represented by the eligible population or those sampled from the eligible 
population. A simple formula of dividing the numerator into the denominator produces the rate 
reported to the SMA and the SPHA, expressed in percentages. There are a number of other 
specifications for sampling, oversampling, replacement, and treatment of contraindications for 
services that are further explained in the HEDIS 2004 Technical Specifications: Volume 2!9 to which 


the interested reader is referred. 


TIME FRAME 

According to the HEDIS technical specifications, the time frame for including members in the eligible 
population or sample was the measurement year of calendar year (CY) 2003 for the Adolescent 
Immunization Status and the Adolescent Well-Care Visits measures. The time frame for including 
members in the eligible population or sample was the measurement year (CY2003) and the year 
before the measurement year (CY2002) for the Use of Appropriate Medications for People with 
Asthma measure. The time frame for the events of interest (e.g., immunizations, well-care visits, 


and use of medication) was CY2003. 


PROCEDURES FOR DATA COLLECTION 

The HEDIS 2004 technical specifications for each measure validated were reviewed by the EQRO 
Project Director, Research Analyst, and a health management informatics consultant. Extensive 
training in data management and programming for healthcare quality indices, clinical training, 
research methods, and statistical analysis expertise were well represented among the personnel 
involved in adapting and implementing the Validating of Performance Measures Protocol to conform 


to the HEDIS, SMA, and SPHA requirements while maintaining consistency with the Validating 


*° National Committee for Quality Assurance (2003). HEDIS 2004, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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Performance Measures Protocol. The following sections describe the procedures for each activity in 
the Validating Performance Measures Protocol as they were implemented for the three HEDIS 2004 


measures validated. 


Pre-On-Site Activity One: Reviewer Worksheets 

Reviewer Worksheets were developed for the purpose of conducting activities and recording 
observations and comments for follow-up at the site visits. HEDIS 2004 technical specifications 
were used to refine the Reviewer Worksheets for the evaluation of each item. Throughout 
September and October 2004, project personnel met weekly to review available source documents 
and develop the Reviewer Worksheets for conducting pre-on-site, on-site, and post-on-site 
activities as described below. The reviews formed the basis for completing the Attachments (V, VII, 
X, Xll, XIll, and XV) of the Validating Performance Measures Protocol for each measure and MC+ 
MCO. Source documents used to develop the methods for review and complete the Attachments 


included: 


e Information Systems Capabilities Assessments (ISCA) developed by the SMA and completed by the MCOs 
during 2003 and 2004 

e HEDIS 2004 Data Submission Tool (DST) 

e HEDIS 2004 Baseline Assessment Tool (BAT) 

e HEDIS 2004 Audit Report 

e HEDIS 2004 SPHA Reports 


Pre-On-Site Activity Two: Preparation of MC+ MCOs 

Individual conversations with QI/UM Coordinator or the Medicaid Plan Administrator designee, 
presentations, MC+ MCO orientation teleconference calls, and individual communications with 
personnel at MC+ MCOs responsible for HEDIS 2004 performance measure calculation were 
conducted between October 2004 and February 2005, with follow-up telephone calls and written 
communications continuing through April 2005. From October 18, 2004 through October 29, 
2004, the EQRO conducted technical assistance orientation phone calls with each of the MC+ 
MCOs to provide education about the Validating of Performance Measures Protocol and the EQRO 
submission requirements. All written materials, letters and instructions were reviewed and 
approved by the SMA in advance. Technical objectives, methods, procedures, data sources, 
communication with the EQRO, and contact information for EQRO personnel were provided to 
MC+ MCOs prior to the teleconference calls. MC+ MCOs were requested to have in attendance 
the person(s) responsible for the calculation of the HEDIS 2004 performance measures validated. 
Teleconference meetings were led by the EQRO Project Director, with key project personnel and a 


representative from the SMA in attendance. Technical assistance was focused on describing the 
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Validating Performance Measures Protocol; identifying the three measures validated; the purpose, 
activities and objectives of the EQRO; and defining the information and data needed for the EQRO 


to validate the performance measures. 


Additional technical assistance was provided to the attendees of the MC+ MCO All-Plan Meeting on 
October 21, 2004 by the EQRO Project Director and personnel. On November 12, 2004, formal 
written requests for data and information from the MC+ MCOs for the validation of performance 
measures were made by the EQRO, to be submitted by MC+ MCOs by November 23, 2004 (see 
Appendix 4). Detailed letters and instructions were mailed to QI/UM Coordinators and Medicaid 
Plan Administrators explaining the type of information, purpose, and format of submissions. EQRO 
personnel were available and responded to electronic mail and telephone inquiries; and requested 
clarification throughout the evaluation process. The following are the data and documents 
requested from MC+ MCOs for the Validating Performance Measures Protocol: 


™ HEDIS 2004 Data Submission Tool for all three measures for the MC+ Managed Care Population only. Do not 


include other measures or populations. 


=" 2004 HEDIS Audit Report. This is the HEDIS Performance Audit Report for the MC+ Managed Care Program 
product line and the three MC+ measures to be validated (complete report). If the three measures to be 
validated were not audited or if they were not audited for the MC+ Managed Care Program population, please 
send the report, as it contains Information Systems Capability Assessment Information that can be used as part 
of the Protocol. 


Baseline Assessment Tool for HEDIS 2004. The Baseline Assessment Tool is to include descriptions of the 
process for calculating measures for the MC+ Managed Care Program population. 


List of cases for denominator with all HEDIS 2004 data elements specified in the measures. 


List of cases for numerators with all HEDIS 2004 data elements specified in the measures, including fields for 
claims data and MOHSAIC, or other administrative data used. Please note that one of the review elements in 
the Protocol is: The “MCO/PIHP has retained copies of files or databases used for performance measure 
reporting, in the event that results need to be reproduced.” 


List of cases for which medical records were reviewed, with all HEDIS 2004 data elements specified in the 
measures. BHC will request MCOs to gather a maximum of 30 records per measure, based on a random sample 


and the MCO will be requested to send copies. 


Sample medical record tools used for hybrid methods for the three HEDIS 2004 measures for the MC+ Managed 
Care Program population; and instructions for reviewers. 


All worksheets, memos, minutes, documentation, policies and communications within the MCO and with HEDIS 
auditors regarding the calculation of the selected measures. 


Policies, procedures, data and information used to produce numerators and denominators. 
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Policies, procedures, and data used to implement sampling (if sampling was used). At a minimum, this should 
include documentation to facilitate evaluation of: 


e = Statistical testing of results and any corrections or adjustments made after processing. 

e Description of sampling techniques and documentation that assures the reviewer that samples used for 
baseline and repeat measurements of the performance measures were chosen using the same sampling 
frame and methodology. 

e Documentation of calculation for changes in performance from previous periods (if comparisons were 
made), including tests of statistical significance. 


Policies and procedures for mapping non-standard codes. 
Record and file formats and descriptions for entry, intermediate, and repository files. 


Electronic transmission procedures documentation. (This will apply if MCO sends or receives data electronically 
from vendors performing the HEDIS abstractions, calculations or data entry.) 


Descriptive documentation for data entry, transfer, and manipulation programs and processes. 


Samples of data from repository and transaction files to assess accuracy and completeness of the transfer 
process. 


Documentation of proper run controls and of staff review of report runs. 


Documentation of results of statistical tests and any corrections or adjustments to data along with justification 
for such changes. 


Documentation of sources of any supporting external data or prior years’ data used in reporting. 


Procedures to identify, track, and link member enrollment by product line, product, geographic area, age, sex, 
member months, and member years. 


Procedures to track individual members through enrollment, disenrollment, and possible re-enrollment. 
Procedures used to link member months to member age. 


Documentation of “frozen” or archived files from which the samples were drawn, and if applicable, 
documentation of the MCO’s process to re-draw a sample or obtain necessary replacements. 


=" Procedures to capture data that may reside outside the MCO’s data sets (e.g. MOHSAIC). 


Policies, procedures, and materials that evidence proper training, supervision, and adequate tools for medical 
record abstraction tasks. (May include training material, checks of inter-rater reliability, etc.) 


Pre-On-Site Activity Three: Assess the Integrity of the MCO's Information 
System 


The objective of this activity was to assess the integrity of the MC+ MCOs’ ability to link data from 

multiple sources. Once the Reviewer Worksheets were developed, EQRO personnel reviewed the 
SMA-developed and administered Information Systems Capability Assessment (ISCA) and the HEDIS 
2004 Baseline Assessment Tool (BAT) submitted by each MC+ MCO. Detailed notes and follow-up 


questions were formulated for the site visit review. 


On-Site Activity One: Assess Data Integration and Control 


39 
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The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple sources 
and assess whether these abilities ensure the accuracy of the measures. The site visit activities 
addressed a series of technical, process, and competency reviews with personnel (including 
management and technical staff) and vendors involved in the development and production of the 
HEDIS 2004 performance measures. The site visit activities examined the HEDIS 2004 reporting 
processes, databases, software and vendors for the three measures validated. This included 
reviewing data processing issues for generating the rates and determining the numerator and 
denominator counts. Other activities involved generating ad-hoc reports based on similar criteria 
for at least one of the measures, reviewing database processing systems, software, organizational 
reporting structures, and sampling methods. The following are the activities conducted at each 
MC+ MCO: 


e Review results of run queries (on-site observation, screen-shots, test output) 

e Examination of data fields for numerator & denominator calculation (examine field definitions and file 
content) 

e Review of applications, data formats, flowcharts, edit checks and file layouts 

e Review of source code, software certification reports 

e Review HEDIS repository procedures, software manuals 

e Test for code capture within system for measures (confirm principal & secondary codes, presence/absence 
of non-standard codes) 

e Review of operating reports 

e Review information system policies (data control, disaster recovery) 

e Review vendor associations & contracts 


The following are the interview questions developed for the site visits: 


e What are the processes of data integration and control within information systems? 

e What documentation processes are present for collection of data, steps taken and procedures to calculate 
the HEDIS measures? 

e What processes are used to produce denominators? 

e What processes are used to produce numerators? 

e How is sampling done for calculation of rates produced by the hybrid method? 

e How does the MCO submit the requirement performance reports to the State? 


From the site visit activities, interviews, and document reviews, Attachment V (Data Integration and 
Control Findings) of the Protocol was completed for each MC+ MCO and performance measure 


validated. 


On-Site Activity Two: Assess Documentation of Data and Processes Used to 
Calculate and Report Performance Measures 


The objectives of this activity were to assess the documentation of data collection, assess the 
process of integrating data into a performance measure set, and examine procedures used to query 


the data set to identify numerators, denominators, generate a sample, and apply proper algorithms. 
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From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment VII (Data and Processes Used to Calculate and Report Performance) of the 
Protocol was completed for each MC+ MCO and measure validated. One limitation of this step 
was the inability of MC+ MCOs to provide documentation of processes used to calculate and report 
the performance measures due to the use of proprietary software or off-site vendor software and 


claims systems. 


On-Site Activity Three: Assess Processes Used to Produce the Denominators 
The objectives of this activity were to determine the extent to which all eligible members were 
included, evaluate programming logic and source codes relevant to each measure, and evaluate 


eligibility, enrollment, age, codes, and specifications related to each performance measure. 


The content and quality of the data files submitted were reviewed to facilitate the evaluation of 
compliance with the HEDIS 2004 technical specifications. MC+ MCOs did not consistently submit 
the requested level of data (e.g., all elements required by the measures or information on hybrid or 
administrative data). The fields in the data files sometimes included information such as member 
identification number, member name, date of birth, enrollment start and end dates, dates of 
services, and diagnosis or procedure codes for each of the three measures. The format and content 
of files submitted by MC+ MCOs varied such that it precluded a standard method for evaluating and 
validating the numerators and denominators across MC+ MCOs. There was little to no 
documentation of the data files, the field names, the definitions of the field names, or the processes 
of determining numerators and denominators contained in the files submitted. The format of the 
data files varied across MC+ MCOs depending on the MC+ MCO’s information system design and 
the HEDIS reporting software output. Thus, a general approach was developed after evaluation of 
the content and format of numerator and denominator files submitted by all MC+ MCOs. Specific 
methods for each MC+ MCO were based on the actual content and format of the data provided by 
individual MC+ MCOs. Additional information regarding the format and content of files was 
gathered through communication with the designated contact person at each MC+ MCO, the 
QI/UM Coordinator, and site visit interviews with personnel and vendors responsible for calculating 


and reporting the HEDIS 2004 performance measures. 


The validation of denominators in the submitted files involved examining the range of values for each 
of the criteria, such as dates of birth of members, and the last dates of enrollment. The number of 


cases that were designated as denominators and met the denominator criteria for the valid ranges 
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were considered valid. From the site visit activities, interviews, review of numerator and 
denominator files and document reviews, Attachment X (Denominator Validation Findings) of the 
Protocol was completed for each MC+ MCO and performance measure validated. 

On-Site Activity Four: Assess Processes Used to Produce the Numerators 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to accurately identify medical 
events (e.g., immunizations, well-care visits, medications), evaluate the MCOs’ ability to identify 
events from other sources (e.g., medical records, State Public Immunization Registry), assess the use 
of codes for medical events, evaluate procedures for non-duplication of event counting, examine 
time parameters, review the use of non-standard codes and maps, identify medical record review 
procedures (Hybrid Method), and review the process of integrating administrative and medical 


record data. 


For the Administrative Method, validation of the numerators was conducted for all three measures 
using the specified parameters for the dates of service(s), diagnosis codes, procedure codes, and 
pharmacy codes as appropriate to the respective measures. For example, for all three measures, 
dates of service were required to occur between January |, 2003 and December 31, 2003. Cases 


with dates outside this range were considered not valid. 


Validation of numerators for the Hybrid Method followed the Validating Performance Measures 
Protocol for sample selection and calculation of bias related to the medical record review. The 
Protocol requires the EQRO to sample up to 30 records from the medical records reported by the 
MC+ MCO as meeting the numerator criteria (hybrid hits). In the event that the MC+ MCO 
reported fewer than 30 numerator events from medical records, the EQRO requested all medical 
records that were reported by the MC+ MCO as meeting the numerator criteria. This approach 
does not apply to the Use of Appropriate Medications for People with Asthma, as the 


Administrative Method of calculation is required by HEDIS technical specifications. 


In some cases, the number of cases in the numerator files were not able to be validated against 
those reported on the HEDIS DST. For six of the seven MC+ MCOs, it was discovered that not all 
possible records up to the sample of 30 were initially requested by the EQRO as a result of the 
inability to identify numerator cases in the numerator files. Initial requests for documents and data 
were made on November 8, 2004, with submissions due to the EQRO by November 23, 2004. 


Samples of medical records were requested from MC+ MCOQs on January 3, 2005. For those 
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medical records that were requested subsequent to the discovery that not all numerators were 
identified (February 4, 2005), the submission deadline was extended to February 18, 2005. There 
was confusion on the part of some MC+ MCOs about the nature of documentation (medical 
records, not printouts of databases or reviewer worksheets) and the measures validated (e.g., 
Combination #1, not Combination #2 for Adolescent Immunization Status). The EQRO continued 
to work closely with several MC+ MCOs and accepted records submitted as late as February 24, 
2005. This allowed an additional fourteen business days from the follow-up request for the 30 or 
fewer sampled records for each of the two measures using the Hybrid Method. The timelines for 
medical record requests of a total of 60 or fewer records per MC+ MCO were within the five day 
time frame specified by the State of Missouri Medicaid Managed Care Contract for providing small 
numbers of medical records. Additionally, the process allowed for re-submission of records for 
MC+ MCOs that provided file layouts without any method of identifying hybrid cases. It should be 
noted that several MC+ MCOs submitted records via fax and electronic mail despite strong 
admonishments from EQRO personnel. MC+ MCOs should not transmit Protected Health 
Information through electronic mail without encryption or de-identification of Limited Data Set 


information. 


The review of medical records was administered by Reliable Health Care, Inc. (RHC), a temporary 
healthcare services provider located in Kansas City, Missouri and a Business Associate of Behavioral 
Health Concepts, Inc., (the EQRO). This company is a State of Missouri certified Minority-Owned 
Business Enterprise (MBE) operated by two registered nurses. RHC possesses expertise in 
recruiting nursing and professional health care staff for clinical, administrative, and HEDIS medical 
record review services. The review of medical records was conducted by RNs with over 20 years 
of clinical experience and who were currently licensed and practicing in the State of Missouri. Two 
RNs participated in the training and medical record review process and both had at least three years 


of experience conducting medical record reviews for HEDIS measures. 


Medical record abstraction tools for the Adolescent Immunization Status, Combination #1, and the 
Adolescent Well-Care Visits measures were developed by the EQRO Project Director and revised 
in consultation with a physician and research analyst and with the input of the nurse reviewers. The 
2004 HEDIS technical specifications and the Validating Performance Measures Protocol criteria were 
used to develop the medical record review tools and data analysis plan. A medical record review 


manual and documentation of ongoing reviewer questions and resolutions were developed for the 
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present review. A full day of training was conducted by the EQRO Project Director on February 
25, 2005 using sample medical record tools and reviewing all responses with feedback and 
discussion. The reviewer training and training manual covered content areas such as Health 
Insurance Portability and Accountability Act (HIPAA), confidentiality, conflict of interest, review 
tools, project background, Missouri’s Early and Periodic Screening, Diagnosis, and Treatment 
Program (EPSDT; the Healthy Children and Youth; HCY Program) and forms, Association for 
Professionals in Infection Control and Epidemiology (APIC) guidelines, and Bright Futures Guidelines 
(promulgated by the American Academy of Pediatricians). Teleconference meetings between the 
nurses, coders, and EQRO Project Director were conducted at least weekly, and additionally as 
needed to resolve questions and coding discrepancies throughout the duration of the medical 


record review process. 


A Senior Research Associate with three years of experience conducting EQRO medical record 
review data collection and analysis and five years of experience collecting, managing, and analyzing 
data using relational database and specialized statistical software packages managed the process of 
medical record data collection and analysis. A data entry format with validation parameters was 
developed for accurate data entry. A data entry manual and a full day of training was provided to 
the data entry person at RHC, Inc. Data were reviewed weekly for accuracy and completeness, 
with feedback and corrections made to the data entry person. The final databases were reviewed 
for validity, verified, and corrected prior to performing analyses. All data analyses were developed, 
reviewed, approved, and finalized by the EQRO Project Director. Attachments XIl (Impact of 
Medical Record Findings) and XIll (Numerator Validation Findings) were completed based on the 


medical record review of documents and site visit interviews. 


On-Site Activity Five: Assess Sampling Process (Hybrid Method) 
The objective of this activity was to assess the representativeness of the sample of care provided. 


e Review Information Systems Capability Assessment (ISCA) 

e Review HEDIS Baseline Assessment Tool (BAT) 

e Review Data Submission Tool (DST) 

e Review numerator and denominator files 

e Conduct medical record review for measures calculated using hybrid methodology 

e Determine the extent to which the record extract files are consistent with the data found in the medical 
records 

e Review of medical record abstraction tools and instructions 

e Conduct on-site interviews, activities, and review of additional documentation 

e For those MCOs that calculated the Adolescent Immunization or Adolescent Well Care Visit measures via 
hybrid methodology, a sample of medical records (up to 30) was conducted to validate the presence of 
immunizations that contributed to the numerator. 
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From the review of documents and site visits, Attachment XV (Sampling Validation Findings) was 
completed for those MC+ MCOs that elected the Hybrid Method for the HEDIS 2004 Adolescent 


Immunization Status, Combination #1 and the HEDIS 2004 Adolescent Well-Care Visits measure. 


On-Site Activity Six: Assess Submission of Required Performance Measures to 
State 


The objective of this activity was to assure proper submission of findings to the SMA and SPHA. 
The DST was obtained from the SPHA to determine the submission of the performance measures 
validated. Conversations with the SPHA representative responsible for compiling the measures for 
all MCOs in the State occurred with the EQRO Project Director to clarify questions, obtain data, 


and follow-up on MC+ MCO submission status. 


Post- On-Site Activity One: Determine Preliminary Validation Findings for 
each Measure 


Calculation of Bias 

The Validating Performance Measures Protocol specifies the method for calculating bias based on 
medical record review for the Hybrid Method. In addition to examining bias based on the medical 
record review and the Hybrid Method, the EQRO calculated bias related to the inappropriate 
inclusion of cases with administrative data that fell outside the parameters described in the HEDIS 
2004 technical specifications. For measures calculated using the Administrative Method, the EQRO 
examined the numerators and denominators for correct date ranges for dates of birth and dates of 
service as well as correct enrollment periods and codes used to identify the medical events. This 
was conducted as described above under on-site activities three and four. The estimated bias in the 
calculation of the HEDIS 2004 measures for the Hybrid Method was calculated using the following 
procedures, methods and formulas, consistent with the Validating Performance Measures Protocol. 


Specific analytic procedures are described in the following section. 


Analysis 

Once the medical record review was complete, all administrative data provided by the MC+ MCOs 
in their data file submissions for the HEDIS 2004 Adolescent Immunization Status, Combination #| 
measure were combined with the medical record review data collected by the EQRO. This allowed 
for calculation of the final rate by the EQRO for the multiple events (measles, mumps, rubella; MMR; 
and Hepatitis B vaccinations, Hep B). The next step was to remove the duplicate immunizations 


documented in both the medical record and in administrative data. Then, the number of MMR and 
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Hep B vaccinations at the specified age ranges (3 MMRs on or between the members Ist and | 3th 
birthday; 3 Hep Bs on or before the members’ | 3th birthday) were counted for each record. Cases 
that met both criteria were counted as “hits” and considered valid numerators based on medical 


record review. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, only the medical record findings from 
the EQRO were used for analysis since this was a single event. In order for the single event of an 
adolescent well-care visit to be met, there had to be documented evidence of all three components 
specified in the HEDIS 2004 technical specifications (a health and developmental history, physical 


exam, and health education/anticipatory guidance) for there to be a medical record “hit”. 


For the calculation of bias based on medical record review for the MC+ MCOs using the Hybrid 
Method for the HEDIS 2004 Adolescent Immunization Status, Combination #| and Adolescent 
Well-Care Visits measures, several steps were taken. First, the number of hits based on the medical 
record review were reported (Medical Records Validated by EQRO). Second, the Accuracy 
(proportion of Medical Records Validated by EQRO/Numerator Hits by Medical Records reported 
by the MCO) and Error Rates (100% - Accuracy Rate) were determined. Third, A Weight of Each 
Medical Record was calculated (100%/Denominator reported by the MCO) as specified by the 
Protocol. The number of False Positive Records was calculated (Error Rate * Numerator Hits by 
Medical Records reported by the MCO). This represents the number of records that were not able 
to be validated by the EQRO. The Estimated Bias from Medical Records was calculated (False 


Positive Rate * Weight of Each Medical Record). 


To calculate the Total Estimated Bias in the calculation of the performance measures, the 
Administrative Hits Validated by the EQRO (through the previously described file validation 
process) and the Medical Record Hits Validated by the EQRO (as described above) were summed 
and divided by the Denominator reported by the MCO on the DST to determine the Rate Validated 
by the EQRO. The difference between the Rate Validated by the EQRO and the Rate Reported by 
the MC+ MCO to the SMA and SPHA was the Total Estimated Bias. A positive number reflects an 


overestimation of the rate, while a negative number reflects an underestimation of the rate. 


Once the EQRO concluded its on-site activities, the validation activity findings for each performance 
measure were aggregated. This involved the review and analysis of findings and Attachments 


produced for each performance measure selected for validation and for the MCO’s IS as a result of 
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pre-on-site and on-site activities. The EQRO Project Director reviewed and finalized all ratings 
before submitting to the SMA for final ratings on all Attachments, and completed the Final 
Performance Measure Validation Worksheets for all measures validated and MC+ MCOs. Ratings 
for each of the Worksheet items (0 = Not Met; | = Partially Met ; 2 = Met) were summed for each 
worksheet and divided by the number of applicable items to form a rate for comparison to other 
MC+ MCOs. The worksheets for each measure were examined by the EQRO Project Director to 


complete the Final Audit Rating. 


Below is a summary of the final audit rating definitions specified in the Protocol. Any measures not 
reported were considered “Not Valid.” A Total Estimated Bias outside the 95% upper and lower 
confidence limits of the measures as reported by the MC+ MCO on the DST was considered not 


valid. 


Fully Compliant: Measure was fully compliant with State (SMA and SPHA) specifications. 


Substantially Compliant: Measure was substantially compliant with State (SMA and SPHA) 
specifications and had only minor deviations that did not significantly bias 
the reported rate. 


Not Valid: Measure deviated from State specifications such that the reported rate was 
significantly biased. This designation is also assigned to measures for which 
no rate was reported. Significantly biased was defined by the EQRO as being 
outside the 95% confidence interval of the rate reported by the MC+ MCO on 
the HEDIS 2004 Data Submission Tool. 





Once the EQRO submitted its preliminary findings to the MC+ MCOs, the MC+ MCOs were 
offered the opportunity to submit comments and documentation to support the correction of 
factual errors or omissions in the EQRO’s preliminary report. Several MC+ MCOs submitted 


corrective action plans to address issues identified in the preliminary report. 


Findings 

MC+ MCOs conduct the calculation of performance measures in collaboration with a variety of 
vendors and use a number of different management information systems to extract data for the 
calculation of measures. They are also required to undergo annual audits by NCQA-accredited 
auditing firms that provide MC+ MCOs with recommendations for reporting or not reporting 
findings of specific measures to the NCQA. Regardless of the NCQA audit rating or rotation, MC+ 


MCOs are required to report the performance measures validated to the SMA and SPHA. Table 7 
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summarizes the names of HEDI|S-certified software used, medical record vendors, and HEDIS 
auditors. Tables 8 and 9 show the method of calculation used by each MC+ MCO and the audit 
ratings assigned by the NCQA Auditor. This information was taken from the NCQA-certified 
Auditors’ reports and MC+ MCO self-report to the EQRO. 


Table 7. HEDIS 2004 Software, Vendors, and Auditors for the HEDIS 2004 Measures. 


Name of Medical Name of HEDIS 2004 
MC+ MCO Name of Software Record Vendor PANU Teliwelg 
MS Access, MS Excel Healthcare Research 
Community Care Plus (Novasys) QMark/HEDISHelp Associates 
Health Plan Mercy Health Plan/Q- Healthcare Research 
Mercy Health Plan Reporter* Mark Associates 
Quality Spectrum* 
HEDIS repository by 
Catalyst Not Applicable. Do not 
HealthCare USA Technologies use Hybrid Method. HealthcareData.com, LLC 
Austin Provider 
Missouri Care Solutions Missouri Care Thomson MedStat 
Software from 
Family Health Partners vendor (OAOSH) Family Health Partners Qualis Health 
Health Plan 
FirstGuard Reporter* FirstGuard Metastar 
Health Plan 
Blue Advantage Plus Reporter* QMark/HEDISHelp Ernst & Young 
Note: * NCQA-certified. 





Table 8. Summary of Method of Calculation Reported and Validated by MC+ MCOs. 
Adolescent 
Immunization UFY= Meo} wy AV eo) eo) ge) eo) a t-1a-) 
Status, Adolescent Well- Medications for 

MC+ MCO Combination #1 Care Visits People with Asthma 
Community Care Plus Hybrid Administrative Administrative 
Mercy Health Plan Hybrid Hybrid Administrative 
HealthCare USA Hybrid Administrative Administrative 
Missouri Care Hybrid Hybrid Administrative 
Family Health Partners Hybrid Administrative Administrative 
FirstGuard Hybrid Hybrid Administrative 
Blue Advantage Plus Not Calculated Administrative Administrative 





Table 9. Audit Designations from NCQA-Certified Auditors. 
Adolescent 
Immunization URY= Meo} mV eo) eo) ae) ela ti a-) 
Status Adolescent Medications for 
Ob a Audit Type Combination #1 Well-Care Visits People with Asthma 
Community Care Plus Partial 
Mercy Health Plan Full 


HealthCare USA Partial 


Missouri Care Partial 

Family Health Partners Partial 

FirstGuard Full 

Blue Advantage Plus Full 
Note: NA = Measure not audited; NR = Measure not reported; R = Measure reportable; NCQA = National Committee 
for Quality Assurance. 
Source: MCO self-report and NCQA Audit Report for HEDIS 2004. 
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The validation of each of the performance measure is discussed in the following sections with the 
findings from each validation activity described. Subsequent sections summarize the status of 
submission of the measures validated to the SMA and SPHA, the Final Audit Ratings, and 


conclusions. 


HEDIS 2004 ADOLESCENT IMMUNIZATION STATUS, COMBINATION #1 

Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple 
sources. It is based on the integrity of the management information systems and the ability to 
ensure accuracy of the measures. For the HEDIS 2004 Adolescent Immunization Status, 
Combination #1 measure, the sources of data included enrollment, eligibility, claim files, and State 
Public Health Immunization Registry (MOHSAIC) data. Table 10 summarizes the findings of 
Attachment V (Data Integration and Control Findings) of the protocol. The rate of items that were 


met was calculated across MC+ MCOs and from the number of applicable items for each MC+ 


MCO. 


Of all the MC+ MCOs that calculated the measure, 84.6% Met the criteria for having accurate and 
established procedures for transferring data into data repositories for calculation of the measure, 
coordinating data integration with vendors, and following standards associated with programming 
and testing. All criteria were Met by six of the seven MC+ MCOs that calculated the measure 
(85.7%) except the one for documentation and reporting of software specifications for measure 
calculation. One MC+ MCO (Community Care Plus) Partially Met the criteria for documentation of 
the reporting software program with respect to all aspects of performance measure reporting 
(71.4%). Each MC+ MCO calculating the measure Met 92.3% to 100.0% of the criteria for data 


integration and control. 
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Table 10. Data Integration and Control Findings, HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 





All MC+ MCOs 


MC+ MCO | 
Number 
Partially [Number Not} Number | Rate 
Audit Elements BA+ Met Met Applicable} Met 


MCO/PIHP processes accurately and completely transfer data from the 
transaction files (e.g., membership, provider, encounter/claims) into the 
repository used to keep the data until the calculations of the performance 
5.1 measures have been completed and validated. 2 2 2 2 2 2 0 6 0 1 d 85.7% 


5.2 Samples of data from repository are complete and accurate. 2 2 2 2 2 2 0 6 0 i 7 85.7% 


MCO’s/PIHP’s processes to consolidate diversified files, and to extract required 
5.3. information from the performance measure repository are appropriate. 2 2 2 2 2 2 0 6 0 1 7 85.7% 


Actual results of file consolidations or extracts were consistent with those 
which should have resulted according to documented algorithms or 

5.4 specifications. 2 2 2 2 Z 2 0 6 0 il 7 85.7% 
Procedures for coordinating the activities of multiple subcontractors ensure the 
accurate, timely, and complete integration of data into the performance 

5.5 measure database. 2 2 2 2 2 2 0 6 0 1 7 85.7% 
Computer program reports or documentation reflect vendor coordination 
activities, and no data necessary to performance measure reporting are lost or 


5.6 inappropriately modified during transfer. 2 2 2 2 2 2 0 6 0 il 7 85.7% 
The repository’s design, program flow charts, and source codes enable 

5.7 analyses and reports. 2 2 2 2 2 2 0 6 0 1 7 85.7% 
Proper linkage mechanisms have been employed to join data from all 

5.8 necessary sources (e.g., identifying a member with a given disease/condition). 2 2 2 2 2 2 0 6 0 il 7 85.7% 


Examine and assess the adequacy of the documentation governing the 
production process, including MCO/PIHP production activity logs, and 


5.9 MCO/PIHP staff review of report runs. 2 2 2 2 2 2 0 6 0 1 7 85.7% 

5.10 Prescribed data cutoff dates were followed. 2 2 2 2 2 2 0 6 0 il 7 85.7% 
The MCO/PIHP has retained copies of files or databases used for performance 

5.11 measure reporting, in the event that results need to be reproduced. 2 2 2 2 2 2 0 6 0 1 7 85.7% 


Review documentation standards to determine the extent to which the 
reporting software program is properly documented with respect to every 
aspect of the performance measurement reporting repository, including 
5.12 building, maintaining, managing, testing, and report production. il Z Z Zz Z Z 0 5 il il 7 71.4% 


Review the MCO’s/PIHP’s processes and documentation to determine the 
extent to which they comply with the MCO/PIHP standards associated with 


5.13 reporting program specifications, code review, and testing. 2 2 2 2 2 2 0 6 0 cl 7 85.7% 
Number Met 12 13 13 13 13 13 O (xz 1 13 91 84.6% _ 
Number Partially Met i 0 0 0 0 0 0 
Number Not Met 0 0 0 0 0 0 13 
Number Applicable 13 13 13 13 13 13 13 
Rate Met 92.3% 100.0% 100.0% 100.0% 100.0% 100.0% 0.0% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling numerators and denominators; and the ability to apply proper algorithms. Table | | 
summarizes the findings of Attachment VI (Data and Processes Used to Calculate and Report 
Performance Measures) of the Protocol. Item 7.3 (Statistical testing of results and corrections made 
after processing) did not apply to the measure. Items 7.5, 7.7, and 7.10 were not applicable to 
HealthCare USA, as they relate only to the use of the Hybrid Method of calculation. MC+ MCOs 
Met 61.3% of the criteria for applying appropriate data and process for the calculation of the HEDIS 
2004 Adolescent Immunization Status, Combination #1 measure. Of the MC+ MCOs that 
calculated the measure, 66.3% Met the criteria for documentation of data and processes. All MC+ 
MCOs that calculated the measure (six of seven; 85.7%) Met the criteria for following data file and 
field definitions. Five of the six MC+ MCOs used the Hybrid Method for calculation, and four Met 
criteria (66.7%) for having detailed medical record review practices and reviewer training materials. 
One MC+ MCO (Missouri Care) was unable to locate medical record review tools due to staff 
turnover. One MC+ MCO (Missouri Care) used maps to non-standard coding and Met the criteria 
for this item. External data sources (State Public Health Immunization Registry) for calculation of 
the measure were incorporated by five MC+ MCOs (71.4%). Community Care Plus did not 
incorporate State Public Health Immunization Registry (MOHSAIC) data into the calculation of the 
measure, and Blue Advantage Plus did not calculate the measure. Procedures for sampling were Met 
for five of six MC+ MCOs (83.3%). Although MC+ MCOs frequently graphed the rates of 
performance over several years, none of the MC+ MCOs used the calculation of statistical 
significance in rates from year to year as a measure of the significance of fluctuation in the measure 
(see items 7.8 and 7.11). When sampling, five of the six MC+ MCOs using the Hybrid Method Met 
(83.3%) the criteria for using appropriate statistical functions for determining confidence intervals 
for sampling. Each MC+ MCO calculating the measure Met 66.7% to 77.8% of the criteria for 


documentation of data and processes. 
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Table |1. Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2004 Adolescent Immunization Status, Combination #| 
Measure. 


MC+ MCO All MC+ MCOs 


Number 
Partially Number 
Audit Elements CCP BA+ Met Not Met | Applicable |Rate Met 
2 2 2 2 2 2 0 6 1 





Data file and field definitions used for each measure. 7 85.7% 
Maps to standard coding if not used in original data collection. NA NA NA 2 NA NA 0 il 0 il 2 50.0% 
Statistical testing of results and any corrections or adjustments made after 

7.3. processing. NA NA NA NA NA NA NA 0 0 0 0 NA 
All data sources, including external data (whether from a vendor, public 

7.4 — registry, or other outside source), and any prior years’ data (if applicable). 0 2 2 2 2 22 0 5 0 2 i 71.4% 


Detailed medical record review methods and practices, including the 
qualifications of medical record review supervisor and staff; reviewer 
training materials; audit tools used, including completed copies of each 
record-level reviewer determination; all case-level critical performance 
measure data elements used to determine a positive or negative event or 
exclude a case from same; and inter-rater reliability testing procedures and 


7.5 results. 2 2 NA 0 2 2 0 4 0 2 6 66.7% 
Detailed computer queries, programming logic, or source code used to 
7.6 identify the population or sample for the denominator and/or numerator. 2 2 2 2 2 2 0 6 0 1 Ui 85.7% 


If sampling used, description of sampling techniques, and documentation 
that assures the reviewer that samples used for baseline and repeat 
measurements of the performance measures were chosen using the same 


7.7. sampling frame and methodology. 2 2 NA 2 2 2 0 5 0 1 6 83.3% 
Documentation of calculation for changes in performance from previous 
7.8 periods (if applicable), including statistical tests of significance. 0 0 0 0 0 0 0 0 0 7 Vi 0.0% 

















Data that are related from measure to measure are consistent (e.g., 
7.9 membership counts, provider totals, number of pregnancies and births). 2 2 2 2 2 2 0 6 0 a 7 85.7% 


Appropriate statistical functions are used to determine confidence intervals 
7.10 when sampling is used in the measure. 2 2 NA 2 2 2 0 5 0 1 6 83.3% 


When determining improvement in performance between measurement 
periods, appropriate statistical methodology is applied to determine levels 


7.11 of significance of changes. 0 0 0 0 0 0 0 
Number Met 6 7 4 7 7 7 0 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 3 2 2 3 2 2 10 
Number Applicable 9 9 6 10 9 9 10 
Rate Met 66.7% 77.8% 66.7% 70.0% 77.8% 77.8% 0.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for calculating each measure. Table |2 summarizes the findings of Attachment X 
(Denominator Validation Findings) of the protocol. Items 10.5 (Identification of gender of the 
member), 10.6 (Calculation of member months or years), and 10.10 (Systems for estimating 
populations when they are unable to accurately be counted) were not applicable to this measure. 
Of the MC+ MCOs that calculated the measure, 85.7% Met the criteria for producing denominators 
according to specifications. This was due to one MC+ MCO not calculating measure. Six of the 
seven MC+ MCOs (85.7%) calculating the measure Met all criteria for the production of 
denominators. Each MC+ MCO calculating the measure Met the criteria for processes used to 


produce the denominators 100.0% of the time. 
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Table 12. Denominator Validation Findings, HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 
All MC+ MCOs 


MC+ MCO 
Number 
; Partially Number Number Rate 
Audit Elements BA+ | Met Not Met |Applicable| Met 


All members who were eligible to receive the specified services were 
included in the initial population from which the final denominator was 
produced. This "at risk" population included both members who received 
the services, as well as those who did not. This same standard applies to 
provider groups or other relevant populations identified in the specifications 
10.1 of each performance measure. 2 2 2 2 2 2 0 6 0 1 Es 85.7% 


For each measure, programming logic or source code which identifies, 
tracks, and links member enrollment within and across product lines (e.g., 
Medicare and Medicaid), by age and sex, as well as through possible 
periods of enrollment and disenrollment, has been appropriately applied 





10.2 according to the specifications of each performance measure. 2 2 2 2 2 2 0 6 0 i ii 85.7% 
Calculations of continuous enrollment criteria were correctly carried out and 

10.3 applied to each measure (if applicable). 2 2 2 2 2 2 0 6 0 1 7 85.7% 
Proper mathematical operations were used to determine patient age or 

10.4 range. Z Z 2 2 2 Z 0 6 0 1 7 85.7% 


The MCO/PIHP can identify the variable(s) that define the member's sex in 
every file or algorithm needed to calculate the performance measure 
denominator, and the MCO/PIHP can explain what classification is carried 


10.5 out if neither of the required codes is present.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP has correctly calculated member months and member years, 
10.6 if applicable to the performance measure.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP has properly evaluated the completeness and accuracy of 
any codes used to identify medical events, such as diagnoses, procedures, 
or prescriptions, and these codes have been appropriately identified and 
10.7 applied as specified in each performance measure. 2 2 2 Ps 2 2 0 6 0 1 7 85.7% 


Any time parameters required by the specifications of the performance 
measure are followed (e.g., cut off dates for data collection, counting 30 
10.8 calendar days after discharge from a hospital, etc.). 2 2 2 2 2 2 0 6 0 i 7 85.7% 


Performance measure specifications or definitions that exclude members 
from a denominator were followed. For example, if a measure relates to 
receipt of a specific service, the denominator may need to be adjusted to 
reflect instances in which the patient refuses the service or the service is 
10.9 contraindicated. 2 2 2 2 2 2 0 6 0 1 7 85.7% 


Systems or methods used by the MCO/PIHP to estimate populations when 
they cannot be accurately or completely counted (e.g., newborns) are 


10.10 valid.* NA NA NA NA NA NA NA 0 0 0 0 NA 
Number Met 7 7 7 7 7 7 0 42 (0) 7 49 85.7% 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 0 0 0 0 0 7 
Number Applicable y | 7 7 7 7 7 


Rate Met 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 0.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | =Partially Met; O = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Figure | illustrates the rate of eligible members per MC+ MCO, based on the enrollment of all MC+ 
MCO Waiver Recipients as of December 31, 2003, the end of the CY2003 measurement year. It 
was expected that MC+ MCOs would identify similar proportions of eligible members for the 
measure. The rate of eligible members (percent of eligible members divided by the total 
enrollment) was calculated for all MC+ MCOs. Two-tailed z-tests of each MC+ MCO comparing 
MC+ MCOs to the rate of eligible members for all MC+ MCOs were conducted at the 95% level of 
confidence. Family Health Partners (3.40%) and FirstGuard (3.07%) identified significantly higher 
rates of eligible members than the rate for all MC+ MCOs (2.86%), while Mercy Health Plan (2.04%) 
identified a significantly lower rate of eligible members. The difference in rates may be due to the 
demographic characteristics of the member population, the completeness of claims data, or the 
processes of identifying eligible members. The identification of eligible members for the HEDIS 2004 
Adolescent Immunization Status, Combination #| is dependent on the quality of the enrollment and 
eligibility files. 


Figure |. MC+ Managed Care Program HEDIS 2004 Adolescent Immunization Status, Combination #1, 
Eligible Members. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. Enrollment as of the last week in December 2003 (the measurement year) was used to 
calculate the rate. 

Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); Missouri Department of Social Services, Division of Medical Services, 
State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2003. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non- 
duplicate counting of multiple events, review time parameters and the use of non-standard code 
maps, and assess the processes and procedures for collecting and incorporating medical record 
review data. Table 13 shows the numerators, denominators, rates, and confidence intervals 
submitted by the MC+ MCOs to the SPHA on the DST for the HEDIS 2004 Adolescent 
Immunization Status, Combination #| measure. The rate for all MC+ MCOs was calculated by the 
EQRO, thus, there is no confidence interval reported for the statewide rate. The rate for all MC+ 
MCOs was 14.36%, with MC+ MCO rates ranging from 0.00% (Blue Advantage Plus) to 58.88% 


(Family Health Partners). 
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Table 13. Data Submission for HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 


Final Data 
Co} | [=e a fe) a DY=Vakolagliat- ne) s 
| (Or amy (OL) Method Used (DST) 


Community Care Plus — Hybrid 411 


Mercy Health Plan Hybrid 411 
HealthCare USA Administrative 5,492 
Missouri Care Hybrid 411 


Family Health 
Partners Hybrid 


FirstGuard Hybrid 
Blue Advantage Plus Not Calculated 


All MC+ MCOs 


Administrative Hits i (Yel Ker.) mp X-kere) ge, Total Hits 
Reported by MCO Hits Reported Reported by 
(DST) by MCO (DST) MCO (DST) 
143 144 


62 


Rate Reported 
by MCO (DST) 


35.04% 


19.22% 
4.79% 


52.07% 
58.88% 


34.55% 
0.00% 


14.36% 


Uo ee ee) 
35.30% - 
39.77% 

15.29% - 
23.15% 


NA 
47.12% - 
57.02% 
54.00% - 
63.76% 
29.83% - 
39.27% 


0.00% 





Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); LCL = 95% Lower Confidence 
Limit; UCL = 95% Upper Confidence Limit. The denominator is either the eligible population (for administrative method calculation) or the sample size (for hybrid method 
calculation). The EQRO is charged with providing MCO to MCO comparisons. Therefore, the numerators and denominators for HealthCare USA were aggregated across all 
three regions, which were each 100% validated. Mercy Health Plans submitted data for Combination #2; Blue Advantage Plus did not calculate the measure. The statewide rate 
for all MC+ MCOs was calculated by the EQRO using the sum of numerators divided by the sum of denominators. There was no statewide rate or confidence limits reported to 


the SMA or SPHA. 


Source: MC+ Managed Care Organization HEDIS 2004 Data Submission Tools (DST). 
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Figures 2, 3, and 4 illustrate the rates reported by the MC+ MCOs and the rates of administrative 
and hybrid hits for each MC+ MCO on the HEDIS 2004 Adolescent Immunization Status, 
Combination #| measure. The rate reported by each MC+ MCO was compared with the rate for 
all MC+ MCOs, with two-tailed z-tests conducted at the 95% confidence interval. The rate for all 
MC+ MCOs was lower than the National Commercial (55.93%) and the National Medicaid rates 
(58.70%). Family Health Partners and Missouri Care reported rates significantly higher than the rate 
for all MC+ MCOs (58.88% and 52.07%, respectively), while HealthCare USAs combined rate 
(4.79%) across all three regions was significantly below the rate for all MC+ MCOs (14.36%). 


Figure 2. MC+ Managed Care Program HEDIS 2004 Adolescent Immunization Status, Combination #1. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 


significance, two-tailed z-test. 
Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); National Committee for Quality Assurance (NCQA). 


When the rates of administrative and hybrid hits were examined separately (see Figures 3 and 4), 
Family Health Partners and Missouri Care reported significantly higher rates of administrative hits 
(32.60% and 15.33%, respectively) than the rate for all MC+ MCOs (6.73%). However, HealthCare 


USA and Community Care Plus identified significantly lower rates of administrative hits (4.79% and 
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24%, respectively) than the rate for all MC+ MCOs. This may be a function of the completeness of 
each MC+ MCOs claim system or claims for adolescent immunizations. A possible reason for the 
low rate of administrative hits by Community Care Plus is the exclusion of the State Public Health 


Immunization Registry (MOHSAIC) data in calculating the numerators. 


Figure 3. MC+ Managed Care Program HEDIS 2004 Adolescent Immunization Status Rates, Combination 
#1, Administrative Rate Only. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 


significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2004 Data Submission Tool (DST). 


For hybrid hits, Community Care Plus and Missouri Care reported significantly higher rates of 
hybrid hits based on medical record review (34.79% and 36.74%, respectively) than the rate for all 
MC+ MCOs (28.03%) using the Hybrid Method. Mercy Health Plan reported a significantly lower 
rate of hybrid hits based on medical record review (15.09%) than the rate calculated across all MC+ 
MCOs. Differences may be due to the differences in processes for carrying out medical record 


reviews and compiling hybrid data to calculate the rate using the Hybrid Method. 


59 


Performance Management Solutions Group 
8. HC a Division of Behavioral Health Concepts, Inc. 
2716 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MISSOURI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Figure 4. MC+ Managed Care Program HEDIS 2004 Adolescent Immunization Status Rates, Combination 
#1, Hybrid Rate Only. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2004 Data Submission Tool (DST). 


Tables 14 and 15 summarize the findings of the EQRO medical record review validation and 
Attachment XII (Impact of Medical Record Findings) of the Protocol. Five of the MC+ MCOs used 
the Hybrid Method of calculation. Each MC+ MCO selected a sample of 411 eligible members, 
consistent with HEDIS technical specifications. A total of 147 of the 576 medical records reported 
as hybrid hits by MC+ MCOs were sampled for validation by the EQRO. Only those records 
received were included in the validation. Of the 147 medical records sampled, 106 were received 
for review (72.11%), and 93 were able to be validated (63.27%), resulting in an error rate of 36.73% 
across all MC+ MCOs using the Hybrid Method of calculation. The number of False Positive 
Records (the total amount that could not be validated) was 212 of the 576 reported hits. The 
estimated bias for individual MC+ MCOs based on the medical record validation ranged from a 
|.68% to 20.88% overestimate in the rate, with an estimated bias of 10.30% for all MC+ MCOs using 


the Hybrid Method. Table 15 shows the impact of the medical record review findings. The error 
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rate ranged from I 1.11% to 60.00%, with a rate of 36.73% for all MC+ MCOs using the Hybrid 
Method. The final estimated bias in the final rate ranged from 3.65% to 20.92%, with an average of 


3.59% for all MC+ MCOs after taking into account the validation of administrative hits (see Item 


12.8, Table 15). 


Table 16 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Items 13.8 through 13.13 relate to the Hybrid Method and were not 
applicable to HealthCare USA. Across all MC+ MCOs, 80.0% of the criteria for calculating the 
numerator were met. Six of the seven (85.7%) MC+ MCOs calculating the measures Met criteria 
for using the appropriate data to identify the at-risk population, using complete medical event codes, 
correctly classifying members for inclusion in the numerator, using consistent non-standard code 
maps, following time parameters for the specification of the measure, adequately training record 
review staff, and using appropriate notation for medical record reviews for the measured event. 
Five of the seven (71.4%) MC+ MCOs Met criteria for capturing data for performance indicators 
that could be easily underreported due to services delivered outside the MC+ MCO. Community 
Care Plus did not meet the criteria due to the exclusion of MOHSAIC data. Each MC+ MCO 


calculating the measure Met 91.7% to 100.0% of the criteria for processes used to produce the 


numerators. 
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Table 14. Medical Record Validation for HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 


Number Number 
Numerator i C=te fer] Medical Number 
Hits by AX =Tele) ge f-} AX =tole] ge f- Medical Rate Weight of Estimated 
Denominator}! Medical |Sampled for|/Received for| Records’ | Validated of =t- Lola] False Bias from 
(Sample AX=Tole} gre Audit by Audit by |Validated by| Records {|Accuracy| Error Medical /Positive| Medical 
MC+ MCO SIP 4=)) (DST) [=e], fe) =e} {e) =e], {e) Received Rate Rate X=reo) ge Records| Records 


Community Care Plus 411 143 30 13 12 92.31% 40.00% 60.00% 0.002 86 20.88% 


Mercy Health Plan 411 62 2H 26 24 92.31% 88.89% 11.11% 0.002 7 1.68% 
Missouri Care 411 151 30 23 15 65.22% 50.00% 50.00% 0.002 76 18.37% 
Family Health 

Partners 108 30 29 27 93.10% 90.00% 10.00% 0.002 11 2.63% 
FirstGuard al 30 15 15 100.00% 50.00% 50.00% 0.002 56 13.63% 

All MC+ MCOs 87.74% 63.27% 36.73% 0.0005 212 10.30% 
Note: DST = Data Submission Tool; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); Accuracy Rate = Number of Medical Records Validated 

by the EQRO/Number of Records Selected for Audit by EQRO; Error Rate = 100% - Accuracy Rate; Weight of Each Medical Record = Error Rate * Medical Record Hits 

Reported by MCO; Estimated Bias from Medical Records = Percent of bias due to the medical record review = False Positive Rate * Weight of Each Medical Record.. 

Source: MC+ MCO Data Submission Tools (DST); BHC, Inc. 2004 External Quality Review Performance Measures Validation. 





Table 15. Impact of Medical Record Findings, HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 





MC+ MCO 
Audit Elements ccp_ | MHP_| HcUSA | MOCare | FHP | FG 
12.1 Final Data Collection Method Used (e.g., MRR, hybrid,) H H A H H H Not Calculated 
12.2 Error Rate (Percentage of records selected for audit that were identified as not meeting numerator requirements) 60.00% 11.11% NA 50.00% 20.00% 50.00% 0.00% 
|12.3 Is error rate < 10%? (Yes or No) No No NA No No No 0 | 
If yes, MCO/PIHP passes MRR validation; no further MRR calculations are necessary. NA NA NA NA NA NA 0 
If no, the rest of the spreadsheet will be completed to determine the impact on the final rate. See Below See Below NA See Below See Below See Below 0 
Denominator (The total number of members identified for the denominator of this measure, as identified by the 
12.4 MCO/PIHP) 411 411 NA 411 411 411 0 
Weight of Each Medical Record (Impact of each medical record on the final overall rate; determined by dividing 
12.5 100% by the denominator) 0.002 0.002 NA 0.002 0.002 0.002 0 
12.6 Total Number of MRR Numerator Positives identified by the MCO/PIHP using MRR. 143 62 NA ileyl 108 2 0 
Expected Number of False Positives (Estimated number of medical records inappropriately counted as numerator 
12.7 positives) 86 7 NA 76 11 56 0 
Estimated Bias in Final Rate (The amount of bias caused by medical record review and review of numerator and 
12.8 denominator files) 20.92% Basove NA 18.98% 3.65% 17.03% 0 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Table 16. Numerator Validation Findings, HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 
All MC+ MCOs 





MC+ MCO 
Number 
’ Partially | Number Number 
Audit Elements CCP BA+ Met Not Met | Applicable 


The MCO/PIHP has used the appropriate data, including linked data from separate 

13.1 data sets, to identify the entire at-risk population. 2 2 2 2 2 2 0 6 0 1 7 85.7% 
The MCO/PIHP has in place and utilizes procedures to capture data for those 
performance indicators that could be easily under-reported due to the availability of 


13.2 services outside the MCO/PIHP. 0 2 2 2 2 2 0 5 0 2 7 71.4% 
The MCO's/PIHP's use of codes used to identify medical events are complete, 

13.3 accurate, and specific in correctly describing what has transpired and when. 2 2 2 2 2 2 0 6 0 1 7 85.7% 
The MCO/PIHP correctly evaluated medical event codes when classifying members 

13.4 for inclusion or exclusion in the numerator. 2 2 2 2 2 2 0 6 0 1 7 85.7% 
The MCO/PIHP has avoided or eliminated all double-counted members or numerator 

13.5 events. 2 2 2 2 2 2 0 6 0 1 7 85.7% 


Any non-standard codes used in determining the numerator have been mapped to a 
standard coding scheme in a manner that is consistent, complete, and reproducible 
as evidenced by a review of the programming logic or a demonstration of the 
13.6 program. NA NA NA 2 NA NA 0 it 0 il 2 50.0% 
Any time parameters required by the specifications of the performance measure are 
adhered to (i.e., that the measured event occurred during the time period specified 


13.7 or defined in the performance measure). 2 2 2 2 2 2 0 6 0 Il 7 85.7% 
Medical record reviews and abstractions have been carried out in a manner that 

13.8 facilitates the collection of complete, accurate, and valid data. 2 0 NA 1 2 2 0 3 1 2 6 50.0% 

13.9 Record review staff have been properly trained and supervised for the task. Z 2 NA 2 2 2 0 5 0 i 6 83.3% 
Record abstraction tools require the appropriate notation that the measured event 

13.10 occurred. 2 Z NA 2 Z 2 0 5 0 1 6 83.3% 
Record abstraction tools require notation of the results or findings of the measured 

13.11 event (if applicable). 2 2 NA 2 2 2 0 5 0 il 6 83.3% 


Data included in the record extract files are consistent with data found in the 
medical records as evidenced by a review of a sample of medical record for 
applicable performance measures. (From Medical Record Review Validation Tools- 


13.12 Table 5, ATTACHMENT XII) 2 2 NA 2 2 2 0 5 0 i! 6 83.3% 
The process of integrating administrative data and medical record data for the 

13.13 purpose of determining the numerator is consistent and valid. 2 2 NA 2 2 2 0 5 0 1 6 83.3% 
Number Met I Ii 6 12 12 12 Oo (iy! 1 15 80 80.0% 
Number Partially Met 0 iL. 0 0 
Number Not Met 1 1 0 0 0 13 
Number Applicable 12 12 6 13 12 12 13 
Rate Met 91.7% 91.7% 100.0% 92.3% 100.0% 100.0% 0.0% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Sampling Procedures for Hybrid Method 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to randomly sample from the 
eligible members for the measure when using the Hybrid Method of calculation. Table |7 summarizes 
the findings of Attachment XV (Sampling Validation Findings) of the Protocol. Item 15.3 (Each provider 
had an equal chance of being sampled) was not applicable to the HEDIS 2004 Adolescent Immunization 
Status, Combination #1 measure; and none of the items were applicable to HealthCare USA. Item 15.9 
(Documenting if the requested sample size exceeded the eligible population size) did not apply to any of 
the MC+ MCOs for this measure. MC+ MCOs Met criteria (81.7%) for random sampling without 
systematic exclusion, examining files for bias, assuring there was no correlation between samples drawn, 
assuring members had the same chance of being included at baseline and follow-up measurement, 
maintaining sample files, meeting sample size requirements of the performance measure specifications, 
oversampling to accommodate for exclusions, and making substitutions properly. The criteria for 
sample exclusions was Met by four out of six (66.7%) of the MC+ MCOs. Mercy Health Plan 
systematically excluded medical records sampled for the Hybrid Method by omitting those that did not 
have a record of a claim in the past three years from the medical record review. Of the MC+ MCOs 


that calculated the measure, the rate for proper sampling ranged from 90.0% to 100.0%. 
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Table 17. Sampling Validation Findings, HEDIS 2004 Adolescent Immunization Status, Combination #1 Measure. 
MC+ MCO All MC+ MCOs 





Number 
Partially | Number 
Audit Elements HCUSA BA+ Met Coy im (= AV 8) eI et- 19) (=) 


Each relevant member or provider had an equal chance of being selected; no one 
15.1 was systematically excluded from the sampling. 2 2 NA 2 2 2 0 5 0 ul 6 83.3% 


The MCO / PIHP followed the specifications set forth in the performance measure 
regarding the treatment of sample exclusions and replacements, and if any activity 
took place involving replacements of or exclusions from the sample, the MCO/PIHP 


15.2 kept adequate documentation of that activity. Z 6) NA 2 Zz Z 0 4 0 Z 6 66.7% 
Each provider serving a given number of enrollees had the same probability of being 
15.3 selected as any other provider serving the same number of enrollees.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP examined its sampled files for bias, and if any bias was detected, the 
MCO/PIHP is able to provide documentation that describes any efforts taken to 


15.4 correct it. 2 Z NA Z Z Z 0 5 0 i 6 83.3% 
The sampling methodology employed treated all measures independently, and there 
15.5 is no correlation between drawn samples. 2 2 NA 2 2 2 0 5 0 a 6 83.3% 


Relevant members or providers who were not included in the sample for the 
baseline measurement had the same chance of being selected for the follow-up 
15.6 measurement as providers who were included in the baseline. 2 2 NA 2 2 2 0 5 0 il 6 83.3% 
The MCO/PIHP has policies and procedures to maintain files from which the samples 
are drawn in order to keep the population intact in the event that a sample must be 
re-drawn, or replacements made, and documentation that the original population is 


15.7 intact. Z Z NA 2 Z Z 0 5 0 1 6 83.3% 

15.8 Sample sizes meet the requirements of the performance measure specifications. 2 2 NA 2 2 2 0 5 0 it 6 83.3% 
The MCO/PIHP has appropriately handled the documentation and reporting of the 

15.9 measure if the requested sample size exceeds the population size. NA NA NA NA NA NA NA 0 0 0 0 NA 

15.10 The MCO/PIHP properly oversampled in order to accommodate potential exclusions 2 2 NA 2 2 2 0 5 0 il 6 83.3% 









Substitution applied only to those members who met the exclusion criteria specified 
15.11 in the performance measure definitions or requirements. 2 2 NA 2 2 2 0 5 0 1 6 83.3% 


Substitutions were made for properly excluded records and the percentage of 





15.12 substituted records was documented. 2 
Number Met 9 0 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 1 0 0 0 0 10 
Number Applicable 10 10 0 10 10 10 10 
Rate Met 100.0% 90.0% NA 100.0% 100.0% 100.0% 0.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Submission of Measures to the State 

Reports from the SPHA were obtained regarding the submission of the HEDIS 2004 Adolescent 
Immunization Status Combination #1 measure. Six of the seven (85.7%) MC+ MCOs calculated and 
submitted the measure to the SPHA and SMA. Blue Advantage Plus did not calculate or report the 
measure to the SPHA, making it non-compliant with this requirement. The NCQA Audit Report 
indicated that “the MCO did not calculate the measure.” Reasons given for not calculating the 
measure obtained at the site visit were due to the rate being too low, and having been advised by 
the NCQA Auditor to not report the measure. Regardless of the NCQA requirements or 
recommendations, all HMOs in the State of Missouri are required to calculate and report the 


measure to the SPHA, and MC+ MCOs are required to report the measure to the SMA. 


Final Validation Findings 

Table 18 shows the final data validation findings and the total estimated bias in calculation based on 
the validation of medical record data and review of the MC+ MCO extract files for calculating the 
HEDIS 2004 Adolescent Immunization Status, Combination #1 measure. Figure 5 illustrates the 
differences between the rates reported to the SPHA and those calculated by the EQRO. The rate 
for all MC+ MCOs calculated based on data validated by the EQRO was 10.77%, while the rate 
reported by MC+ MCOs was 14.36% (see Table |3 and Figure 2), a 3.59% overestimate. 


Table 18. Final Data Validation for HEDIS 2004 Adolescent Immunization Status, Combination #1 
Measure. 
Medical 
Administrative False Record Hits Total Hits Rate Total 
Hits Validated Positive Validated by | Validated Validated Estimated 
(Os am 1 (©) ohms 10) 2X0) X={ eo) ge [ 0] Xe) by EQRO oh’ 10], 46) Bias 


Community Care Plus 57 14.11% 20.92% 
Mercy Health Plan 55 13.87% 5.35% 


HealthCare USA NA 4.79% 0.00% 
Missouri Care 33.09% 18.98% 
Family Health Partners 55.23% 3.65% 
FirstGuard T7a52 7% 17.03% 
Blue Advantage Plus 0.00% 0.00% 


All MC+ MCOs 10.77% 3.59% 
Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral 
Health Concepts, Inc.); LCL = 95% Lower Confidence Limit; UCL = 95% Upper Confidence Limit; False Positive Records 
= Error Rate * Medical Record Hits Reported by MC+ MCO; Medical Record Hits Validated by the EQRO = Medical 
Record Hits Reported by MC+ MCO (DST) - False Positive Records; Total Estimated Bias = Rate Validated by EQRO 
using medical record review and data extract file review - Rate Reported by MC+ MCO (DST). Positive numbers 
represent an overestimate. The EQRO is charged with providing MCO to MCO comparisons. Therefore, the numerators 
and denominators for HealthCare USA were aggregated across all three regions, which were each 100% validated. Mercy 
Health Plans submitted data for Combination #2; Blue Advantage Plus did not calculate the measure. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Figure 5. Rates Reported by MC+ MCOs and Validated by EQRO, HEDIS 2004 Adolescent Immunization 
Status, Combination #1. 
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Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); BHC, Inc., 2004 External Quality Review Performance Measure 
Validation. 


HEDIS 2004 ADOLESCENT WELL-CARE VISITS 

Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple sources 
for the calculation of the HEDIS 2004 Adolescent Well-Care Visits measure. It is related to the 
integrity of the management information systems and the ability to ensure accuracy of the measures. 
For the HEDIS 2004 Adolescent Well-Care Visits measure, the sources of data included enrollment, 
eligibility, and claim files. Table 19 summarizes the findings of Attachment V (Data Integration and 
Control Findings) of the Protocol. The rate of items that were Met was calculated across MC+ 


MCOs and from the number of applicable items for each MC+ MCO. 


Across all MC+ MCOs, 95.6% of the criteria for data integration and control were Met. All MC+ 
MCOs (100.0%) Met the criteria for having accurate and established procedures for transferring 
data into data repositories for calculation of the measure, coordinating data integration with 
vendors, and following standards associated with programming and testing. One MC+ MCO 
(Community Care Plus) did not meet the criteria for submitting extract files consistent with 


specifications, for using prescribed cutoff dates, or producing accurate file extracts for the measure 
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(CY 2002 rather than CY2003 events were used for calculating and reporting the measure). 
Community Care Plus Partially Met criteria for adequacy of documentation of the production 
process and proper documentation of software with regard to performance measure calculation. 
Community Care Plus Met 69.2% of the criteria for data integration and control, while all other 


MC+ MCOs Met all criteria. 
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Table 19. Data Integration and Control Findings, HEDIS 2004 Adolescent Well-Care Visits Measure. 


5.1 
Did 


5.3 


5.4 


oe, 


5.6 


ene) 


5.8 


Audit Elements 


MCO/PIHP processes accurately and completely transfer data from the 
transaction files (e.g., membership, provider, encounter/claims) into the 
repository used to keep the data until the calculations of the performance 
measures have been completed and validated. 

Samples of data from repository are complete and accurate. 
MCO’s/PIHP’s processes to consolidate diversified files, and to extract 
required information from the performance measure repository are 
appropriate. 

Actual results of file consolidations or extracts were consistent with those 
which should have resulted according to documented algorithms or 
specifications. 

Procedures for coordinating the activities of multiple subcontractors 
ensure the accurate, timely, and complete integration of data into the 
performance measure database. 

Computer program reports or documentation reflect vendor coordination 
activities, and no data necessary to performance measure reporting are 
lost or inappropriately modified during transfer. 


The repository’s design, program flow charts, and source codes enable 
analyses and reports. 

Proper linkage mechanisms have been employed to join data from all 
necessary sources (e.g., identifying a member with a given 

disease/ condition). 

Examine and assess the adequacy of the documentation governing the 
production process, including MCO/PIHP production activity logs, and 
MCO/PIHP staff review of report runs. 

Prescribed data cutoff dates were followed. 

The MCO/PIHP has retained copies of files or databases used for 
performance measure reporting, in the event that results need to be 
reproduced. 

Review documentation standards to determine the extent to which the 
reporting software program is properly documented with respect to every 
aspect of the performance measurement reporting repository, including 
building, maintaining, managing, testing, and report production. 

Review the MCO’s/PIHP’s processes and documentation to determine the 
extent to which they comply with the MCO/PIHP standards associated 
with reporting program specifications, code review, and testing. 


Number Met 

Number Partially Met 
Number Not Met 
Number Applicable 
Rate Met 


MC+ MCO 


2 
9 
2 
2 


13 


69.2% 


13 


13 


100.0% 


13 


13 


100.0% 


13 


13 


100.0% 


13 


13 


100.0% 


13 


13 


100.0% 


2 7 
Zz 7 
2 7 
2 6 
2 7 
2 7 
Z 7 
Zz 7 
2 6 
2 6 
2 7 
2 6 
2 7 
13 87 
0 

0 

13 

100.0% 


Number 


Partially 


Met 


All MC+ MCOs 


Number Number 
Not Met | Applicable 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling, numerators and denominators; and the ability to apply proper algorithms for the 
calculation of HEDIS 2004 Adolescent Well-Care Visits measure. Table 20 summarizes the findings 
of Attachment VI (Data and Processes Used to Calculate and Report Performance Measures) of the 
Protocol. Items 7.3 (Statistical testing of results and corrections made after processing), 7.4 
(Inclusion of external data sources), and 7.9 (Consistent data from measure to measure) did not 
apply to the measure. Across all MC+ MCOs, 60.5% of the criteria were met. All MC+ MCOs Met 
the criteria for following data file and field definitions (100.0%). Three of the seven MC+ MCOs 
(42.9%) used the Hybrid Method for calculation, and two of the three MC+ MCOs (66.7%) Met 
criteria for having detailed medical record review practices and reviewer training materials. One 
MC+ MCO (Missouri Care) was unable to locate medical record review tools due to staff turnover. 
One MC+ MCO (Missouri Care) used maps to non-standard coding and Met the criteria for this. 
Source codes and programming logic for the identification of denominators appeared accurate for all 
MC+ MCOs (100.0%). All three (100.0%) MC+ MCOs using the Hybrid Method Met the criteria for 
documentation of sampling procedures. Although MC+ MCOs frequently graphed the rates of 
performance over several years, none of the MC+ MCOs used the calculation of statistical 
significance in rates from year to year as a measure of the significance of fluctuation in the measure 
(see items 7.8 and 7.11). When sampling, all three (100.0%) of the MC+ MCOs using the Hybrid 
Method Met the criteria for using appropriate statistical functions for determining confidence 
intervals for sampling. Each MC+ MCO calculating the measure Met 50.0% to 71.4% of the criteria 


for processes used to calculate and report the HEDIS 2004 Adolescent Well-Care Visits measure. 
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Table 20. Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2004 Adolescent Well-Care Visits Measure. 





MC+ MCO All MC+ MCOs 
Number 
; Partially | Number 
Audit Elements BA+ Met Not Met | Applicable | Rate Met 
7.1 Data file and field definitions used for each measure. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
7.2 Maps to standard coding if not used in original data collection. NA NA NA 2 NA NA NA i 0 0 il 100.0% 
Statistical testing of results and any corrections or adjustments made after 
7.3 processing. NA NA NA NA NA NA NA 0 0 0 0 NA 
All data sources, including external data (whether from a vendor, public 
7.4 — registry, or other outside source), and any prior years’ data (if applicable). NA NA NA NA NA NA NA 0 0 0 0 NA 
Detailed medical record review methods and practices, including the 
qualifications of medical record review supervisor and staff; reviewer training 
materials; audit tools used, including completed copies of each record-level 
reviewer determination; all case-level critical performance measure data 
elements used to determine a positive or negative event or exclude a case 
7.5 from same; and inter-rater reliability testing procedures and results. NA 2 NA 0 NA 2 NA 2 0 1 3 66.7% 
Detailed computer queries, programming logic, or source code used to identify 
7.6 the population or sample for the denominator and/or numerator. 2 2 2 2 2 2 2 7 0 0 7. 100.0% 
If sampling used, description of sampling techniques, and documentation that 
assures the reviewer that samples used for baseline and repeat measurements 
of the performance measures were chosen using the same sampling frame and 
7.7. ~=methodology. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Documentation of calculation for changes in performance from previous periods 
7.8 (if applicable), including statistical tests of significance. 0 0 0 0 0 0 0 0 0 a 7 0.0% 
Data that are related from measure to measure are consistent (e.g., 
7.9 membership counts, provider totals, number of pregnancies and births) .* NA NA NA NA NA NA NA 0 0 0 0 NA 
Appropriate statistical functions are used to determine confidence intervals 
7.10 when sampling is used in the measure. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
When determining improvement in performance between measurement 
periods, appropriate statistical methodology is applied to determine levels of 
7.11 significance of changes. 0 0 0 0 0 0 0 0 0 7 7 0.0% 
Number Met 2 5 2 5 2 5 2 |e) i) 15 38 60.5% 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 2 2 2 3 2 2 2 
Number Applicable 4 7 4 8 4 Zz 4 


Rate Met 50.0% 71.4% 50.0% 62.5% 50.0% 71.4% 50.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for each measure. For the HEDIS 2004 Adolescent Well-Care Visits measure, the 
sources of data include enrollment, eligibility, and claim files. Table 21 summarizes the findings of 
Attachment X (Denominator Validation Findings) of the protocol. Items 10.5 (Identification of 
gender of the member), 10.6 (Calculation of member months or years), and 10.10 (Systems for 
estimating populations when they are unable to accurately be counted) were not applicable to the 
HEDIS 2004 Adolescent Well-Care Visits measure. Each MC+ MCO calculating the measure Met 


the criteria for processes used to produce the denominators. 
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Table 21. Denominator Validation Findings, HEDIS 2004 Adolescent Well-Care Visits Measure. 
All MC+ MCOs 


MC+ MCO 
Number 
; Partially | Number Number Rate 
Audit Elements WW lelerla BA+ Met Not Met |Applicable| Met 


All members who were eligible to receive the specified services were included 
in the initial population from which the final denominator was produced. This 
"at risk" population included both members who received the services, as well 
as those who did not. This same standard applies to provider groups or other 
relevant populations identified in the specifications of each performance 
10.1 measure. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
For each measure, programming logic or source code which identifies, tracks, 
and links member enrollment within and across product lines (e.g., Medicare 
and Medicaid), by age and sex, as well as through possible periods of 
enrollment and disenrollment, has been appropriately applied according to the 





10.2 specifications of each performance measure. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Calculations of continuous enrollment criteria were correctly carried out and 

10.3. applied to each measure (if applicable). 2 2 2 2 2 2 2 7 0 0 7 100.0% 

10.4 Proper mathematical operations were used to determine patient age or range. 2 2 2 Zz 2 2 2 7 0 0 7 100.0% 


The MCO/PIHP can identify the variable(s) that define the member's sex in 
every file or algorithm needed to calculate the performance measure 
denominator, and the MCO/PIHP can explain what classification is carried out 


10.5 if neither of the required codes is present.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP has correctly calculated member months and member years, if 
10.6 applicable to the performance measure.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP has properly evaluated the completeness and accuracy of any 
codes used to identify medical events, such as diagnoses, procedures, or 
prescriptions, and these codes have been appropriately identified and applied 
10.7 as specified in each performance measure. 2 2 2 2 2 2 2 Fi 0 0 F 100.0% 
Any time parameters required by the specifications of the performance 
measure are followed (e.g., cut off dates for data collection, counting 30 
10.8 calendar days after discharge from a hospital, etc.). 2 2 2 2 2 2 2 7 0 0 i 100.0% 
Performance measure specifications or definitions that exclude members from 
a denominator were followed. For example, if a measure relates to receipt of 
a specific service, the denominator may need to be adjusted to reflect 
instances in which the patient refuses the service or the service is 
10.9 contraindicated.* NA NA NA NA NA NA NA 0 0 0 0 NA 


Systems or methods used by the MCO/PIHP to estimate populations when 
10.10 they cannot be accurately or completely counted (e.g., newborns) are valid.* NA NA NA NA NA NA NA 0 0 0 0 NA 


Number Met 42 O O 42 100.0% 
Number Partially Met 
Number Not Met 
Number Applicable 
Rate Met 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Figure 6 illustrates the rate of eligible members identified by each MC+ MCO, based on the 
enrollment of all MC+ MCO Waiver Recipients as of December 31, 2003, the end of the CY2003 
measurement year. It was expected that MC+ MCOs would identify similar proportions of eligible 
members for the HEDIS 2004 Adolescent Well-Care Visits measure. The rate of eligible members 
(percent of eligible members divided by the total enrollment) was calculated for all MC+ MCOs and 
two-tailed z-tests of each MC+ MCO compared to the state rate of eligible members were 
conducted at the 95% level of confidence. There were no significant differences between MC+ 


MCO rates of eligible members for this measure. 


Figure 6. MC+ Managed Care Program HEDIS 2004 Adolescent Well-Care Visits, Eligible Members. 
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Note: Error bars on the y-axis represent 95% confidence intervals; There were no significant differences on two tailed z- 
tests. 

Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); Missouri Department of Social Services, Division of Medical Services, 
State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2003. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non- 
duplicate counting of multiple events, review time parameters and the use of non-standard code 
maps, and assess the processes and procedures for collecting and incorporating medical record 
review data. For the HEDIS 2004 Adolescent Well-Care Visits measure, the sources of data 
included enrollment, eligibility, and claim files. Table 22 shows the numerators, denominators, rates, 
and confidence intervals submitted by the MC+ MCOs to the SPHA on the DST. The rate for all 
MC+ MCOs was calculated by the EQRO, thus there is no confidence interval reported for the 
statewide rate. The rate for all MC+ MCOs was 30.13%, with MC+ MCO rates ranging from 
18.75% (Community Care Plus) to 41.49% (Missouri Care). 
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Table 22. Data Submission for HEDIS 2004 Adolescent Well-Care Visits Measure. 


Final Data Administrative | Hybrid Hits Total Hits Rate 
fore) | C=Toi a foyam (=a alere imme PX-Talelollat-le)a Hits Reported | Reported by | Reported by | Reported by 
MC+ MCO LUFT =Te | (DST) by MCO (DST) | MCO (DST) MCO (DST) MCO (DST) Ul oj ee Ro Ee Gs) 


Community Care Plus Administrative 9,987 1,873 NA 1,873 18.75% 17.98% - 19.52% 
Mercy Health Plan Hybrid 411 87 15 102 24.82% 20.52% - 29.12% 
HealthCare USA Administrative 41,144 13,139 NA 13,139 39.31% NA 

Missouri Care Hybrid 388 125 36 161 41.49% 36.46% - 46.53% 


Family Health Partners Administrative 10,188 3,355 NA 3,355 32.93% 32.01% - 33.85% 
FirstGuard Hybrid 411 123 18 141 34.31% 29.60% - 39.02% 


Blue Advantage Plus Administrative 6,593 2,057 NA 2,057 31.20% 30.07% - 32.33% 


All MC+ MCOs 69,122 20,759 69 20,828 30.13% 

Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); LCL = 95% Lower Confidence 
Limit; UCL = 95% Upper Confidence Limit. The EQRO is charged with providing MCO to MCO comparisons. Therefore, the numerators and denominators for HealthCare 
USA were aggregated across all three regions, which were each 100% validated. Medical Record Hits Validated by EQRO = Weighted number of medical records with validated 
hits. Community Care Plus reported numerator events from Calendar Year 2002 for the HEDIS 2004 reporting year. The statewide rate for all MC+ MCOs was calculated by 
the EQRO using the sum of numerators divided by sum of denominators. There was no statewide rate or confidence limits reported to the SMA or SPHA. 

Source: MC+ Managed Care Organization HEDIS 2004 Data Submission Tools (DST). 
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Figures 7, 8, and 9 illustrate the rates reported by the MC+ MCOs and the rates of administrative 
and hybrid hits for each MC+ MCO. The rate reported by each MC+ MCO was compared with the 
rate for all MC+ MCOs. Two-tailed z-tests of each MC+ MCO comparing MC+ MCOQOs to the rate 
for all MC+ MCOs were calculated at the 95% confidence interval. The rate for all MC+ MCOs was 
lower than the National Commercial (33.93%) and the National Medicaid rates (37.10%). 
HealthCare USA, Missouri Care, and FirstGuard reported rates significantly higher than the rate for 
all MC+ MCOs (39.31%, 41.49% and 34.31% respectively), while Community Care Plus and Mercy 
Health Plan reported rates significantly below the rate for all MC+ MCOs (18.75% and 24.82%, 


respectively). 


Figure 7. MC+ Managed Care Program HEDIS 2004 Adolescent Well-Care Visits Rates. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); National Committee for Quality Assurance (NCQA). 
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When the rate of administrative and hybrid hits was examined separately, there was wide variability 
among MC+ MCOs from the rate for all MC+ MCOs (30.03%). HealthCare USA, Missouri Care, 
Family Health Partners, and Blue Advantage Plus reported rates of administrative hits significantly 
higher than the rate for all MC+ MCOs (39.31%, 32.22%, 32.93%, and 31.20%, respectively). 
Community Care Plus and Mercy Health Plan reported significantly lower rates of administrative hits 
(18.75% and 21.17%, respectively), while the rate found by FirstGuard was consistent with that of 
the rate for all MC+ MCOs. This may be a function of the completeness of each MC+ MCOs’ claim 


system or the administration of claims for well-care visits. 


Figure 8. MC+ Managed Care Program HEDIS 2004 Adolescent Well-Care Visits, Administrative Rate 
Only. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2004 Data Submission Tool (DST). 
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For hybrid hits, Missouri Care reported a significantly higher rate of hits based on medical record 
review (9.28%) than the rate calculated across all MC+ MCOs (5.70%) using the Hybrid Method. 
Differences may be due to differences in the processes for carrying out medical record reviews and 


compiling hybrid data to calculate the rate. 


Figure 9. MC+ Managed Care Program HEDIS 2004 Adolescent Well-Care Visits, Hybrid Rate Only. 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2004 Data Submission Tool (DST). 
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Tables 23 and 24 summarize the findings of the EQRO medical record review validation and 
Attachment XII (Impact of Medical Record Findings) of the Protocol. Three of the MC+ MCOs 
(Mercy Health Plan, Missouri Care, and FirstGuard) used the Hybrid Method of calculation. Two of 
the three selected a sample of 411 eligible members, consistent with HEDIS technical specifications. 
Missouri Care selected a sample of 388 eligible members, for undetermined reasons. A total of 63 
of the 69 medical records (91.3%) reported as hybrid hits by MC+ MCOs were sampled for 
validation by the EQRO. As few as |3 medical records per MC+ MCO were received for review. 
Those not received were unable to be validated. Of the 63 medical records sampled, 56 were 
received for review (88.89%), and 18 were able to be validated (28.57%), resulting in an error rate 
of 71.43% across all MC+ MCOs using the Hybrid Method of calculation. A total of 30 (53.6%) 
medical records Met the criteria for a medical history documented in the medical record; 48 
(85.7%) Met criteria for documentation of a physical examination; and 26 (46.4%) Met criteria for 
documentation of anticipatory guidance. The number of False Positive Records (the total amount 
that could not be validated) was 49 of the 69 reported hits. The error rate ranged from 33.33% to 
93.33%. The estimated bias for individual MC+ MCOs based on the medical record validation 
ranged from a |.46% to 8.66% overestimate in the rate, with an overestimate of 4.07% for all MC+ 
MCOs. Table 24 shows the impact of the medical record review findings. The estimated bias in the 
final rate for MC+ MCOs using the Hybrid Method ranged from 2.68% to 9.02%, with a rate of 
18.75% for all MC+ MCOs after taking into account the validation of administrative hits (see item 


12.8, Table 24). 
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Table 23. Impact of Medical Record Findings, HEDIS 2004 Adolescent Well-Care Visits Measure. 


Number 
Number Medical | Number Medical | Medical Weight of Estimated 
Numerator AX =T ole) ge [-} AX =Tofo) ge [-} Records | Rate Validated Each False Bias from 
Denominator /|Hits by Medical; Sampled for Received for EVI Te Fin -To of Records Accuracy Medical Positive Medical 
MC+ MCO (Sample Size) |Records (DST) | Audit by EQRO | Audit by EQRO | by EQRO Received Rate Error Rate A X=Tof eo] ge | Records Records 


Mercy Health Plan 411 15 15 : 26.67% 73.33% 0.002 11 
Missouri Care 388 36 30 ; 6.67% 93.33% 0.003 
FirstGuard 411 18 18 66.67% 33.33% 





All MC+ MCOs 1,210 69| 63} 56 32.14% | 28.57%| 71.43%) | 
Note: DST = Data Submission Tool; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); Accuracy Rate = Number of Medical Records Validated 
by the EQRO/Number of Records Selected for Audit by EQRO; Error Rate = 100% - Accuracy Rate; Weight of Each Medical Record = 100%/Denominator; False Positive 
Records = Error Rate * Medical Record Hits Reported by MCO; Estimated Bias from Medical Records = Percent of bias due to the medical record review = False Positive Rate 
* Weight of Each Medical Record. 

Sources: MC+ MCO HEDIS 2004 Data Submission Tools; BHC Inc., 2004 External Quality Review Performance Measures Validation. 


Table 24. Medical Record Validation for HEDIS 2004 Adolescent Well-Care Visits Measure. 





MC+ MCO 


12.1 Final Data Collection Method Used (e.g., MRR, hybrid,) A H A H A H A 
Error Rate (Percentage of records selected for audit that were 
12.2 identified as not meeting numerator requirements) NA 73.33% NA 93.33% NA 33.33% NA 
12.3. Is error rate < 10%? (Yes or No) NA No NA No NA No NA 
If yes, MCO/PIHP passes MRR validation; no further MRR 
calculations are necessary. NA NA NA NA NA NA 
If no, the rest of the spreadsheet will be completed to 
determine the impact on the final rate. NA See Below NA See Below NA See Below NA 
Denominator (The total number of members identified for the 
12.4 denominator of this measure, as identified by the MCO/PI HP) NA 411 NA 388 NA 411 NA 


Weight of Each Medical Record (Impact of each medical record 
on the final overall rate; determined by dividing 100% by the 


12.5 denominator in line 4.) NA 0.002 NA 0.003 NA 0.002 NA 
Total Number of MRR Numerator Positives identified by the 
12.6 MCO/PIHP using MRR. NA 15 NA 36 NA 18 NA 


Expected Number of False Positives (Estimated number of 
medical records inappropriately counted as numerator 
12.7 positives) NA i NA 34 NA 6 NA 
Estimated Bias in Final Rate (The amount of bias caused by 
medical record review and review of numerator and 
12.8 denominator files) NA 2.68% NA 9.02% 3.02% 0.73% NA 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Table 25 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Items [3.8 through 13.13 relate to the Hybrid Method and were not 
applicable to Community Care Plus, HealthCare USA, Family Health Partners, or Blue Advantage 
Plus. Across MC+ MCOs, 90.7% of the criteria for calculating numerators were met. All three of 
the MC+ MCOs calculating the measures using the Hybrid Method (Mercy Health Plan, Missouri 
Care, and FirstGuard) Met all criteria (100.0%) for using the appropriate data to identify the at-risk 
population, using complete medical event codes, correctly classifying members for inclusion in the 
numerator, using consistent non-standard code maps, adequately training record review staff, and 
using appropriate notation for medical record reviews for the measured event. Six of the seven 
MC+ MCOs (85.7%) Met the criteria for following time parameters required by the specifications 
for the measure. One of the three MC+ MCOs (33.3%) using the Hybrid Method carried out 
medical record abstractions in a reliable, accurate manner. The exclusion of members from the 
medical record review resulted in a rating of not met for Mercy Health Plan, and Missouri Care 
Partially Met this criteria. One issue was that the process required duplicate entry of data with the 
potential for introducing data entry error. One of the three (33.3%; Mercy Health Plan) MC+ 
MCOs Met the criteria for consistent and valid integration of medical record and administrative 
data, based on the number of medical records validated. The MC+ MCOs Met 80.0% to 100.0% of 


criteria for calculating the numerator for the HEDIS 2004 Adolescent Well-Care Visits measure. 
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Table 25. Numerator Validation Findings, HEDIS 2004 Adolescent Well-Care Visits Measure. 


All MC+ MCOs 





MC+ MCO 
: Number Number Number 
Audit Elements BA+ Partially Met} Not Met | Applicable | Rate Met 


The MCO/PIHP has used the appropriate data, including linked data from 
13.1 separate data sets, to identify the entire at-risk population. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


The MCO/PIHP has in place and utilizes procedures to capture data for 
those performance indicators that could be easily under-reported due to 
13.2 the availability of services outside the MCO/PIHP. NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO's/PIHP's use of codes used to identify medical events are 
complete, accurate, and specific in correctly describing what has transpired 


13.3. and when. 2 2 2 2 2 2 2 if 0 0 i 100.0% 
The MCO/PIHP correctly evaluated medical event codes when classifying 

13.4 members for inclusion or exclusion in the numerator. 2 Z 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP has avoided or eliminated all double- counted members or 

13.5 numerator events. 2 2. 2 2 Z 2 2 7 0 0 7 100.0% 


Any non-standard codes used in determining the numerator have been 
mapped to a standard coding scheme in a manner that is consistent, 
complete, and reproducible as evidenced by a review of the programming 
13.6 logic or a demonstration of the program. NA NA NA 2 NA NA NA 1 0 0 1 100.0% 
Any time parameters required by the specifications of the performance 
measure are adhered to (i.e., that the measured event occurred during the 


13.7 time period specified or defined in the performance measure). 0 2 2 2 2 2 2 6 0 1 7 85.7% 
Medical record reviews and abstractions have been carried out in a manner 

13.8 that facilitates the collection of complete, accurate, and valid data. NA 0 NA 1 NA 2 NA 1 i 1 3 33.3% 
Record review staff have been properly trained and supervised for the 

13.9 task. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Record abstraction tools require the appropriate notation that the 

13.10 measured event occurred. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Record abstraction tools require notation of the results or findings of the 

13.11 measured event (if applicable). NA 2 NA Wd NA 2 NA 3 0 0 3 100.0% 


Data included in the record extract files are consistent with data found in 
the medical records as evidenced by a review of a sample of medical 
record for applicable performance measures. (From Medical Record Review 





13.12 Validation Tools-Table 5, ATTACHMENT XII) NA 2 NA 0 NA 1 NA 1 1 1 3 33.3% 
The process of integrating administrative data and medical record data for 

13.13 the purpose of determining the numerator is consistent and valid. NA Z NA 2 NA 2 NA 3 0 0 3 NA 
Number Met 4 10 5 10 5 10 5 | 
Number Partially Met 0 0 6) 1 0 1 0 
Number Not Met cl 1 0 0 0 0 
Number Applicable 5 1 5 12 5 11 5 
Rate Met 80.0% 90.9% 100.0% 83.3% 100.0% 90.9% 100.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Sampling Procedures for Hybrid Method 

The objectives of this activity were to evaluation the MC+ MCOs’ ability to randomly sample from 
the eligible members for the measure when using the Hybrid Method of calculation. Table 26 
summarizes the findings of Attachment XV (Sampling Validation Findings) of the Protocol. Items 
[5.3 (Each provider had an equal chance of being sampled) and 15.9 (Documenting if the requested 
sample size exceeded the eligible population size) did not apply to any of the MC+ MCOs for this 
measure; and none of the items were applicable to Community Care Plus, HealthCare USA, Family 
Health Partners, or Blue Advantage Plus. Across all MC+ MCOs, the criteria for sampling were Met 
96.7% of the time. MC+ MCOs Met criteria (100.0%) for random sampling without systematic 
exclusion, examining files for bias, assuring there was no correlation between samples drawn, 
assuring members had the same chance of being included at baseline and follow-up measurement, 
maintaining sample files, meeting sample size requirements of the performance measure 
specifications, oversampling to accommodate for exclusions, and making substitutions properly. The 
criteria for sample exclusions was Met by 66.7% of the MC+ MCOs. Mercy Health Plan 
systematically excluded medical records sampled for the Hybrid Method by omitting those that did 
not have a record of a claim in the past three years from the medical record review. The MC+ 
MCOs using the Hybrid Method of calculating the HEDIS 2004 Adolescent Well-Care Visits 


measure Met 90.0% to 100.0% of the criteria for proper sampling. 


SUBMISSION OF MEASURES TO THE STATE 

Reports from the SPHA were obtained regarding the submission of the HEDIS 2004 Adolescent 
Well-Care Visits measure. All MC+ MCOs reported the measure to the SPHA and SMA. 
Community Care Plus re-calculated and reported a corrected rate after receiving feedback from the 


EQRO!!. 


‘* Personal Communication with Robert Petterson, Program Manager, Managed Care Consumer Guides, Center 
for Health Information Management and Evaluation, Missouri Department of Health and Senior Services. 
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Table 26. Sampling Validation Findings, HEDIS 2004 Adolescent Well-Care Visits Measure. 
MC+ MCO All MC+ MCOs 





Number 
Partially | Number 
Item Audit Elements CCP Met Not Met | Applicable |Rate Met 


Each relevant member or provider had an equal chance of being selected; no 
15.1 one was systematically excluded from the sampling. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


The MCO / PIHP followed the specifications set forth in the performance 
measure regarding the treatment of sample exclusions and replacements, and 
if any activity took place involving replacements of or exclusions from the 


15.2 sample, the MCO/PIHP kept adequate documentation of that activity. NA 0 NA 2 NA 2 NA 2 0 iL 3 66.7% 
Each provider serving a given number of enrollees had the same probability of 
15.3. being selected as any other provider serving the same number of enrollees.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP examined its sampled files for bias, and if any bias was 
detected, the MCO/PIHP is able to provide documentation that describes any 


15.4 efforts taken to correct it. NA 2 NA Z NA 2 NA S 0 0 3 100.0% 
The sampling methodology employed treated all measures independently, and 
15.5 there is no correlation between drawn samples. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


Relevant members or providers who were not included in the sample for the 
baseline measurement had the same chance of being selected for the follow- 
15.6 up measurement as providers who were included in the baseline. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
The MCO/PIHP has policies and procedures to maintain files from which the 
Samples are drawn in order to keep the population intact in the event that a 
sample must be re-drawn, or replacements made, and documentation that the 





15.7 ~— original population is intact. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Sample sizes meet the requirements of the performance measure 

15.8 specifications. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
The MCO/PIHP has appropriately handled the documentation and reporting of 

15.9 the measure if the requested sample size exceeds the population size. NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP properly oversampled in order to accommodate potential 

15.10 exclusions NA 22 NA 2 NA 2 NA 3 0 0 3 100.0% 
Substitution applied only to those members who met the exclusion criteria 

15.11 specified in the performance measure definitions or requirements. NA 2 NA Z NA 2 NA 3 0 0 3 100.0% 
Substitutions were made for properly excluded records and the percentage of 

15.12 substituted records was documented. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Number Met 0 9 0 10 0 10 0 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 1 0 0 0 0 0 
Number Applicable 0 10 0 10 0 10 0 
Rate Met NA 90.0% NA 100.0% NA 100.0% NA 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates 
to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in 
documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Final Validation Findings 

Table 27 shows the final data validation findings for the calculation of the HEDIS 2004 Adolescent 
Well-Care Visits measure and the total estimated bias in calculation based on the validation of 
medical record data and review of the MC+ MCO extract files. Figure 10 illustrates the differences 
between the rates reported to the SPHA and those calculated by the EQRO. The numerators for 
Community Care Plus were not valid, as the incorrect measurement year (2002) was used for 
calculating the measure. The rate for all MC+ MCOs calculated based on data validated by the 


EQRO was 26.91%, while the rate reported by MC+ MCOs was 30.13%, a 3.23% overestimate. 


Table 27. Final Data Validation for HEDIS 2004 Adolescent Well-Care Visits Measure. 


Administrative (aXe | for-] Mm a X-rece) we Mim Ke) e-) Mn Pha) Rate Total 
Hits Validated | False Positive | Hits Validated | Validated | Validated | Estimated 
| (On om (OL) oh = 0] 10) |X =Yelo) ae [> | o)V am = 0) 510) ohm 10] 10) | o)V a 1 0) 10) Bias 


Community Care Plus NA NA 0 0.00% 18.75% 
Mercy Health Plan 11 91 22.14% 2.68% 
HealthCare USA 13,139 31.93% 0.00% 
Missouri Care 126 32.47% 9.02% 


Family Health Partners 3,047 29.91% 3.02% 
FirstGuard 138 33.58% 0.73% 


Blue Advantage Plus 2,057 31.20% 0.00% 


All MC+ MCOs Lord Pay 18,598} 26.91% 3.23% 


Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral 
Health Concepts, Inc.); LCL = 95% Lower Confidence Limit; UCL = 95% Upper Confidence Limit; False Positive Records 
= Error Rate * Medical Record Hits Reported by MCO; Medical Record Hits Validated by the EQRO = Medical Record 
Hits Reported by MCO (DST) - False Positive Records; Total Estimated Bias = Rate Validated by EQRO using medical 
record review and data file review - Rate Reported by MCO. Positive numbers represent an overestimate. The EQRO is 
charged with providing MCO to MCO comparisons. Therefore, the numerators and denominators for HealthCare USA 
were aggregated across all three regions, which were each 100% validated. Community Care Plus reported numerator 
events from Calendar Year 2002 for the HEDIS 2004 reporting year. 

Sources: MC+ Managed Care Organization HEDIS 2004 data Submission Tools (DST); BHC, Inc. External Quality Review 
Performance Measure Validation. 
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Figure 10. Rates Reported by MC+ MCOs and Validated by EQRO, HEDIS 2004 Adolescent Well-Care 


Visits. 


lM Rate Reported by MCO (DST) 18.75% 24.82% 39.31% 


CCP MHP HCUSA 


MOCare FHP FG BA+ 


41.49% 32.93% 34.31% 31.20% 


All MC+ 
MCOs 


30.13% 





®™ Rate Validated by EQRO 


Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); BHC, Inc., 2004 External Quality Review Performance Measure 


Validation. 
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HEDIS 2004 USE OF APPROPRIATE MEDICATION FOR PEOPLE WITH ASTHMA 
Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple 
sources. It is based on the integrity of the management information systems and the ability to 
ensure accuracy of the measures. For the HEDIS 2004 Use of Appropriate Medications for People 
with Asthma measure, the sources of data included enrollment, eligibility, and claim files (Pharmacy, 
Medical, Outpatient Hospital and Inpatient claim types from several sources). Table 28 summarizes 
the findings of Attachment V (Data Integration and Control Findings) of the Protocol. The rate of 
items that were Met was calculated across MC+ MCOs and from the number of applicable items for 


each MC+ MCO. 


Across all MC+ MCOs, 98.9% of the criteria were Met for having accurate and established 
procedures for transferring data into data repositories for calculation of the measure, coordinating 
data integration with vendors, and following standards associated with programming and testing. 
One MC+ MCO (Community Care Plus) Partially Met the criteria for documentation of the 
reporting software program with respect to all aspects of performance measure reporting. Each 


MC+ MCO calculating the measure Met 92.3% to 100.0% of the criteria for data integration and 


control. 
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Table 28. Data Integration and Control Findings, HEDIS 2004 Use of Appropriate Medications for People with Asthma Measure. 





Audit Elements 


MCO/PIHP processes accurately and completely transfer data from the transaction 
files (e.g., membership, provider, encounter/claims) into the repository used to 
keep the data until the calculations of the performance measures have been 


oval! completed and validated. 2 2 2 

5.2 Samples of data from repository are complete and accurate. 2 2 2 
MCO’s/PIHP’s processes to consolidate diversified files, and to extract required 

5.3 information from the performance measure repository are appropriate. 2 2 2 
Actual results of file consolidations or extracts were consistent with those which 

5.4 should have resulted according to documented algorithms or specifications. 2 2 2 


Procedures for coordinating the activities of multiple subcontractors ensure the 
accurate, timely, and complete integration of data into the performance measure 
5.5 database. Z 2 2 
Computer program reports or documentation reflect vendor coordination activities, 
and no data necessary to performance measure reporting are lost or 


5.6 inappropriately modified during transfer. 2 2 2 
The repository’s design, program flow charts, and source codes enable analyses 

5.7 and reports. 2 2 2 
Proper linkage mechanisms have been employed to join data from all necessary 

5.8 sources (e.g., identifying a member with a given disease/condition). 2 2 2 


Examine and assess the adequacy of the documentation governing the production 
process, including MCO/PIHP production activity logs, and MCO/PIHP staff review of 
5.9 report runs. 2 2 Z 


5.10 Prescribed data cutoff dates were followed. 2 2 2 


The MCO/PIHP has retained copies of files or databases used for performance 

5.11 measure reporting, in the event that results need to be reproduced. 2 2 2 
Review documentation standards to determine the extent to which the reporting 
software program is properly documented with respect to every aspect of the 
performance measurement reporting repository, including building, maintaining, 

5.12 managing, testing, and report production. i Z 2 
Review the MCO’s/PIHP’s processes and documentation to determine the extent to 
which they comply with the MCO/PIHP standards associated with reporting program 


5.13 specifications, code review, and testing. 2 2 2 
Number Met 12 13 13 
Number Partially Met 1 0 0 
Number Not Met 0 0 0 
Number Applicable 13 13 13 
Rate Met 92.3% 100.0% 100.0% 


13 


13 


100.0% 


13 


13 


100.0% 


13 


13 


100.0% 


13 


100.0% 





All MC+ MCOs 


MC+ MCO 
Number 
Partially | Number | Number Rate 
BA+ Met Not Met |Applicable| Met 


7 0 0 Ei 100.0% 
if 0 0 7 100.0% 
7. 0 0 7 100.0% 
i 0 0 7. 100.0% 
7 0 0 7 100.0% 
Zt 0 0 7 100.0% 
7 0 0 7 100.0% 
cD 0 0 i 100.0% 
7 0 0 7 100.0% 
vi 0 0 7 100.0% 
d 0 0 7 100.0% 
6 1 0 7 85.7% 
7 0 7 100.0% 
90 1 0 91 98.9% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 


Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling, numerators and denominators; and the ability to apply proper algorithms. Table 29 
summarizes the findings of Attachment VI (Data and Processes Used to Calculate and Report 
Performance Measures) of the Protocol. Items 7.3 (statistical testing of results and corrections 
made after processing), 7.7 (sampling techniques), 7.9 (data consistency from measure to measure), 
and 7.10 (appropriate statistical functions for confidence intervals) did not apply to the measure. 
Across all MC+ MCOs, 64.1% of criteria for calculating and reporting performance measures were 
Met. All MC+ MCOs Met the criteria for following data file and field definitions, the integration of 
external data, and demonstration of detailed queries for identifying eligible members (100.0%). One 
MC+ MCO (Missouri Care) used maps to non-standard coding and Met the criteria for this. 
Although MC+ MCOs frequently graphed the rates of performance over several years, none of the 
MC+ MCOs used the calculation of statistical significance in rates from year to year as a measure of 
the significance of fluctuation in the measure (see items 7.8 and 7.11). Each MC+ MCO Met 60.0% 


to 71.4% of the criteria for calculating and reporting performance measures. 
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Table 29. Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2004 Use of Appropriate Medications for People with 
Asthma Measure. 


MC+ MCO All MC+ MCOs 


Number 
Audit Elements CCP HCUSA Applicable Rate Met 
2 2 2 2 2 2 


qed Data file and field definitions used for each measure. 2 7 0 0 7 100.0% 
Ve Maps to standard coding if not used in original data collection. NA NA NA 2 NA NA NA il 0 0 1 100.0% 


Statistical testing of results and any corrections or adjustments made after 
7.3 processing. NA NA NA NA NA NA NA 0 0 0 0 NA 


All data sources, including external data (whether from a vendor, public 
7.4 registry, or other outside source), and any prior years’ data (if applicable). 2 2 2 2 2 2 2 i 0 0 7 100.0% 
Detailed medical record review methods and practices, including the 
qualifications of medical record review supervisor and staff; reviewer 
training materials; audit tools used, including completed copies of each 
record-level reviewer determination; all case-level critical performance 
measure data elements used to determine a positive or negative event or 
exclude a case from same; and inter-rater reliability testing procedures and 
7.5 results. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


Detailed computer queries, programming logic, or source code used to 
7.6 identify the population or sample for the denominator and/or numerator. 2 2 2 2 2 2 2 7 0 0 7 100.0% 





If sampling used, description of sampling techniques, and documentation 
that assures the reviewer that samples used for baseline and repeat 
measurements of the performance measures were chosen using the same 
7.7 Sampling frame and methodology.* NA NA NA NA NA NA NA 0 0 0 0 NA 


Documentation of calculation for changes in performance from previous 
7.8 periods (if applicable), including statistical tests of significance. 0 0 0 0 0 0 0 0 0 7 i 0.0% 


Data that are related from measure to measure are consistent (e.g., 
7.9 membership counts, provider totals, number of pregnancies and births). NA NA NA NA NA NA NA 0 0 0 0 NA 


Appropriate statistical functions are used to determine confidence intervals 
7.10 when sampling is used in the measure. NA NA NA NA NA NA NA 0 0 0 0 NA 


When determining improvement in performance between measurement 
periods, appropriate statistical methodology is applied to determine levels 
7.11 — of significance of changes. 


0 0 
Number Met 3 4 
Number Partially Met 0 0 
Number Not Met 2 2 
Number Applicable 5 6 5 7 5 6 5 
Rate Met 60.0% 66.7% 60.0% 71.4% 60.0% 66.7% 60.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for each measure. For the HEDIS 2004 Use of Appropriate Medication for People 
with Asthma measure, the sources of data include enrollment, eligibility, and claim files. Table 30 
summarizes the findings of Attachment X (Denominator Validation Findings) of the Protocol. Items 
10.5 (Identification of gender of the member), 10.6 (Calculation of member months or years), and 
10.10 (Systems for estimating populations when they are unable to accurately be counted) were not 
applicable to this measure. Each MC+ MCO calculating the measure Met all of the criteria for 


processes used to produce the denominators. 
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Table 30. Denominator Validation Findings, HEDIS 2004 Use of Appropriate Medications for People with Asthma Measure. 


10.1 


10.2 


10.3 


10.4 


10.5 


10.6 


10.7 


10.8 


10.9 


10.10 


Audit Elements 


All members who were eligible to receive the specified services were 
included in the initial population from which the final denominator was 
produced. This "at risk" population included both members who received 
the services, as well as those who did not. This same standard applies to 
provider groups or other relevant populations identified in the 
specifications of each performance measure. 


For each measure, programming logic or source code which identifies, 
tracks, and links member enrollment within and across product lines 
(e.g., Medicare and Medicaid), by age and sex, as well as through 
possible periods of enrollment and disenrollment, has been appropriately 
applied according to the specifications of each performance measure. 


Calculations of continuous enrollment criteria were correctly carried out 
and applied to each measure (if applicable). 

Proper mathematical operations were used to determine patient age or 
range. 

The MCO/PIHP can identify the variable(s) that define the member's sex 
in every file or algorithm needed to calculate the performance measure 
denominator, and the MCO/PIHP can explain what classification is 
carried out if neither of the required codes is present.* 

The MCO/PIHP has correctly calculated member months and member 
years, if applicable to the performance measure.* 

The MCO/PIHP has properly evaluated the completeness and accuracy of 
any codes used to identify medical events, such as diagnoses, 
procedures, or prescriptions, and these codes have been appropriately 
identified and applied as specified in each performance measure. 

Any time parameters required by the specifications of the performance 
measure are followed (e.g., cut off dates for data collection, counting 30 
calendar days after discharge from a hospital, etc.). 

Performance measure specifications or definitions that exclude members 
from a denominator were followed. For example, if a measure relates to 
receipt of a specific service, the denominator may need to be adjusted 
to reflect instances in which the patient refuses the service or the 
service is contraindicated. 


Systems or methods used by the MCO/PIHP to estimate populations 
when they cannot be accurately or completely counted (e.g., newborns) 
are valid. 


Number Met 

Number Partially Met 
Number Not Met 
Number Applicable 
Rate Met 


MC+ MCO 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


NA 


NA 


N OO ON 


100.0% 


49 


All MC+ MCOs 


Number 
Partially | Number Number 
Met Not Met | Applicable 


0 0 7 
0 0 7 
0 0 7 
0 0 7 
0 0 0 
0 0 0 
0 0 7 
0 0 7 
0 0 7 
0 0 0 
LY) ) 49 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Rate 
Met 





100.0% 


100.0% 


100.0% 


100.0% 


NA 


NA 


100.0% 


100.0% 


100.0% 


NA 
100.0% 
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Figure || illustrates the rate of eligible members per MC+ MCO based on the enrollment of all 


MC+ MCO Waiver Recipients as of December 31, 2003, the end of the CY2003 measurement year. 


It was expected that MC+ MCOs would identify similar proportions of eligible members for the 
measure. The rate of eligible members (percent of eligible members divided by the total 
enrollment) was calculated for all MC+ MCOs. Two-tailed z-tests of each MC+ MCO comparing 
each MC+ MCO to the state rate of eligible members for all MC+ MCOs were calculated at the 
95% level of confidence. Family Health Partners identified a significantly higher rate of eligible 
members (4.93%) compared to the rate for all MC+ MCOs (2.52%), while Mercy Health Plan and 
Missouri Care identified a significantly lower rate of eligible members (1.08% and 1.03%, 
respectively). This may be a function of the claims administration process or member 
characteristics. 


Figure Il. MC+ Managed Care Program HEDIS 2004 Use of Appropriate Medications for People with 
Asthma, Eligible Members. 





7.00% — = 


6.00% 


5.00% 


4.00% 


Rate 


3.00% 




















2.00% 


1.00% 




















0.00% 


All MC+ 
GEP MHP HCUSA MOCare FHP FG BA+ 
MCOs 
@ 2004 Rate 1.83% 1.08% 2.31% 1.03% 4.93% 2.10% 2.24% 2.52% 
MC+ MCO 











Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. Enrollment as of the last week in December 2003 (the measurement year) was used to 
calculate the rate. 


Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); Missouri Department of Social Services, Division of Medical Services, 


State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2003. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non- 
duplicate counting of multiple events, review time parameters and the use of non-standard code 
maps, and assess the processes and procedures for collecting and incorporating medical record 
review data. For the HEDIS 2004 Use of Appropriate Medications for People with Asthma 
measure, the procedures for the Hybrid Method did not apply, as HEDIS 2004 technical 


specifications allow only for the use of the Administrative Method of calculating the measure. 


Table 31 shows the numerators, denominators, rates, and confidence intervals submitted by the 
MC+ MCOs to the SPHA on the DST. The rate for all MC+ MCOs was calculated by the EQRO, 
thus there is no confidence interval reported for the statewide rate. The rate for all MC+ MCOs 
was 63.92% and the rate validated by the EQRO was 63.94%, a .02% underestimate. Figure |2 
illustrates the rates reported by the MC+ MCOs. The rate reported by each MC+ MCO was 
compared with the rate for all MC+ MCOs, with two-tailed z-tests conducted at the 95% 
confidence interval to compare each MC+ MCO with the rate for all MC+ MCOs. The rate for all 
MC+ MCOs was lower than the National Commercial (68.98%) and the National Medicaid rates 
(71.50%). Missouri Care, Family Health Partners, FirstGuard, and Blue Advantage Plus reported 
rates significantly higher than the rate for all MC+ MCOs (65.41%, 66.57% , 67.26%, and 67.39% 
respectively), while Community Care Plus reported a rate (53.71%) significantly below the rate for 


all MC+ MCOs. 
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Table 31. Data Submission and Final Data Validation for HEDIS 2004 Use of Appropriate Medications for People with Asthma Measure. 


Number 
Administrative 
Hits Reported by| Rate Reported 


Administrative Rate 


Site lie) (=) Hits Validated by | Validated by | Estimated 


MC+ MCO 
Community Care Plus 
Mercy Health Plan 
HealthCare USA 


Missouri Care 


Family Health Partners 


FirstGuard 
Blue Advantage Plus 


All MC+ MCOs 


Population MCO (DST) 
849 
447 

4,275 
344 

2,429 


895 


788 531 
10,027 6,409 
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by MCO (DST) 


53.71% 


60.63% 


63.32% 


65.41% 


66.57% 


67.26% 


67.39% 


63.92% 


50.30% 


55.99% 


60.23% 


64.67% 


64.13% 


64.05% 


Uo ee el a) 


57.12% 


65.27% 


NA 


70.58% 


68.47% 


70.39% 


70.72% 


el te 


225 
1,617 


604 


531 


=e] te) 


53.71% 


60.63% 


63.32% 


65.41% 


66.57% 


67.49% 


67.39% 


63.94% 


Bias 


0.00% 


0.00% 


0.00% 


0.00% 


0.00% 


-0.22% 


0.00% 


-0.02% 
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Figure 12. MC+ Managed Care Program HEDIS 2004 Use of Appropriate Medications for People with 
Asthma Rates. 
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@ 2004 Rate 53.71% 60.63% 63.32% 65.41% 66.57% 67.26% 67.39% 63.92% 
MC+ MCO 











Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); National Committee for Quality Assurance (NCQA). 


Table 32 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Item |3.2 was not applicable to the HEDIS 2004 Use of Appropriate 
Medications for People with Asthma measure. Items 13.8 through |3.13 relate to the Hybrid 
Method of calculation and were not applicable to the measure. Across all MC+ MCOs, 94.4% of the 
criteria for calculating numerators were met. All MC+ MCOs Met criteria for using the appropriate 
data to identify the at-risk population, using consistent non-standard code maps, avoiding double- 
counting of events, and following time parameters specified for the measure. Six of the seven MC+ 
MCOs (85.7%) Met the criteria for using complete medical event codes and correctly classifying 
members for inclusion in the numerator. Family Health Partners included 308 numerator events 
that were not within the parameters for the measure. The MC+ MCOs Met 60.0% to 100.0% of 


criteria for the calculation of the numerator. 
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Table 32. Numerator Validation Findings, HEDIS 2004 Use of Appropriate Medications for People with Asthma Measure. 
Comme lee) All MC+ MCOs 


Number 
: Partially 
Audit Elements BA+ Met Ko} am (=) AV) e) et] e) (=) 


The MCO/PIHP has used the appropriate data, including linked data from 
13.1 separate data sets, to identify the entire at-risk population. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


The MCO/PIHP has in place and utilizes procedures to capture data for 
those performance indicators that could be easily under-reported due to 
13.2 the availability of services outside the MCO/PIHP.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO's/PIHP's use of codes used to identify medical events are 
complete, accurate, and specific in correctly describing what has 





13.3. ~=transpired and when. 2 2 2 2 1 2 2 6 1 0 7 85.7% 
The MCO/PIHP correctly evaluated medical event codes when classifying 
13.4 © members for inclusion or exclusion in the numerator. 2 2 2 2 1 2 2 6 al 0 7 85.7% 


The MCO/PIHP has avoided or eliminated all double-counted members or 
13.5 numerator events. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Any non-standard codes used in determining the numerator have been 
mapped to a standard coding scheme in a manner that is consistent, 
complete, and reproducible as evidenced by a review of the programming 
13.6 logic or a demonstration of the program. NA NA NA 2 NA NA NA 1 0 0 il 100.0% 


Any time parameters required by the specifications of the performance 
measure are adhered to (i.e., that the measured event occurred during 
13.7. ~~‘ the time period specified or defined in the performance measure). 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Medical record reviews and abstractions have been carried out in a 
manner that facilitates the collection of complete, accurate, and valid 


13.8 data. * NA NA NA NA NA NA NA 0 0 0 0 NA 
Record review staff have been properly trained and supervised for the 

13.9 task.* NA NA NA NA NA NA NA 0 0 0 0 NA 
Record abstraction tools require the appropriate notation that the 

13.10 measured event occurred.* NA NA NA NA NA NA NA 0 0 0 0 NA 
Record abstraction tools require notation of the results or findings of the 

13.11 measured event (if applicable) .* NA NA NA NA NA NA NA 0 0 0 0 NA 


Data included in the record extract files are consistent with data found in 
the medical records as evidenced by a review of a sample of medical 


13.12 record for applicable performance measures.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The process of integrating administrative data and medical record data 

13.13 for the purpose of determining the numerator is consistent and valid.* NA NA NA NA NA NA NA 6) 0 0 0 NA 
Number Met 5 5 5 6 3 5 5 cf! | 2 0 36 94.4% 
Number Partially Met 0 0 0 0 2 0 0 
Number Not Met 0 0 0 0 0 0 0 
Number Applicable 5 5 5 6 5 5 5 


Rate Met 100.0% 100.0% 100.0% 100.0% 60.0% 100.0% 100.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation 
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Submission of Measures to the State 

Reports from the SPHA were obtained regarding the submission of the HEDIS 2004 Use of 
Appropriate Medications for People with Asthma. All seven MC+ MCOs calculated and submitted 
the measure to the SPHA and SMA. The rate of all MC+ MCOs calculated based on data validated 
by the EQRO was 63.94%, consistent with the rate reported by MC+ MCOs (63.92%), with no 


observed bias. 


Final Validation Findings 

The rate reported by MC+ MCOs ranged from 53.71% to 67.39%. Tables 33 through 35 provide 
summaries of ratings across all Protocol Attachments for each MC+ MCO and measure validated. 
The rate of compliance with the calculation of the Adolescent Immunization Status, Combination #| 
measure specifications ranged from 0.0% (Blue Advantage Plus) to 96.1% (Family Health Partners, 
FirstGuard), with a rate of 78.9% across all MC+ MCOs and items. For the calculation of the 
Adolescent Well-Care Visits measure, MC+ MCO compliance with specifications ranged from 75.0% 
(Community Care Plus) to 93.6% (FirstGuard), with a rate of 90.2% across all MC+ MCOs and 
items. For the rate of compliance with specifications for the calculation of the Use of Appropriate 
Medications for People with Asthma measure, the rate ranged from 86.7% (Family Health Partners) 


to 93.9% (Missouri Care), with an average of 92.1% across all MC+ MCOs. 


Table 33. Summary of Attachment Ratings, HEDIS 2004 Adolescent Immunization Status, Combination 
#1 Measure. 


All MC+ MCOs 


All MC+ 
All Audit Elements CCP Telly Win eler-la- BA+ MCOs 
46 47 30 49 49 0 


Number Met 


Number Partially Met 1 0 0 
Number Not Met 4 4 2 


Number Applicable ou! 51 32 

Rate Met 90.2% 92.2% 93.8% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 
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Table 34. Summary of Attachment Ratings, HEDIS 2004 Adolescent Well-Care Visits Measure. 
MC+ MCO 


All Audit Elements 
Number Met 


Number Partially Met 

Number Not Met 

Number Applicable 

Percent Met 75.0% 91.5% 92.9% 89.8% 92.9% 93.6% 92.9% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 





Table 35. Summary of Attachment Ratings, HEDIS 2004 Use of Appropriate Medication for People with 
Asthma Measure. 


MC+ MCO 


27 29 28 31 2 


Number Met 6 


Number Partially Met 0 0 0 2 0 
Number Not Met 2 2 2 2 


Number Applicable 31 30 33 30 31 

Rate Met 90.0% 93.5% 93.3% 93.9% 86.7% 93.5% 93.3% 

Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2004 External Quality Review Performance Measure Validation. 





Table 36 summarizes the final audit ratings for each of the performance measures and MC+ MCOs. 
The final audit findings for each of the measures was based on the evaluation of processes for 
calculating and reporting the measures, medical record review validation findings, and MC+ MCO 
extract files from repositories. The ratings were based on the impact of medical record review 
findings and the degree of overestimation of the rate as validated by the EQRO. The calculation of 
measures was considered invalid if the specifications were not properly followed, or if the rate 
validated by the EQRO fell outside the confidence intervals for the measure reported by the MC+ 
MCOs on the DST. 
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Table 36. Summary of EQRO Final Audit Ratings, HEDIS 2004 Performance Measures. 


_ Use of Appropriate — 





Adolescent Immunization | Adolescent Well-Care | Medications for 
MC+ MCO Status, Combination #1 Visits People with Asthma 
Community Care Plus Not Valid Not Valid Fully Compliant 
Mercy Health Plan Not Valid Not Valid Fully Compliant 
HealthCare USA Fully Compliant Fully Compliant Fully Compliant 
Missouri Care Not Valid Not Valid Fully Compliant 
Family Health Partners Substantially Compliant Substantially Compliant Fully Compliant 
FirstGuard Not Valid Fully Compliant Fully Compliant 
Blue Advantage Plus Not Valid Fully Compliant Fully Compliant 


Note: Not Valid = Measure deviated from Sate (SMA and SPHA) specifications such that the reported rate was significantly 
biased. This designation is also assigned to measures for which no rate was reported. Significantly biased was defined by 
the EQRO as being outside the 95% confidence interval of the rate reported by the MC+ MCO on the HEDIS 2004 Data 
Submission Tool; Substantially Compliant = Measure was substantially complaint with State (SMA and SPHA) specifications 
and had only minor deviations that did not significantly bias the reported rate; Fully Compliant = Measure was fully 
compliant with State (SMA and SPHA) specifications. Blue Advantage Plus did not report the HEDIS 2004 Adolescent 
Immunization Status, Combination #| measure; and Community Care Plus used 2002 numerator events for calculating the 
HEDIS 2004 Well-Care Visits measure. Data from Health Care USA was aggregated across all three regions of operation 
to provide MCO to MCO comparisons. 

Source: BHC Inc., 2004 External Quality Review Performance Measure Validation. 


For the HEDIS 2004 Adolescent Immunization Status, Combination #1 measure, two MC+ MCOs 
(HealthCare USA and Family Health Partners) were Fully or Substantially Compliant with the 
measure specifications. The rates calculated by Community Care Plus, Mercy Health Plan, Missouri 
Care, FirstGuard, and Blue Advantage Plus were unable to be validated by the EQRO. The rates for 
Community Care Plus, Missouri Care and FirstGuard calculated by the EQRO were outside the 
range of the confidence intervals reported by these MC+ MCOs. The high error rates found on 
medical record review validation (50.00 — 60.00%) were likely due to the low rates of submission of 
medical records for validation (50% or lower). Community Care Plus did not incorporate 
administrative data from the SPHA, and this likely accounted for 8-25% of the error associated with 
calculating the measure. Mercy Health Plan did not follow specifications for the Hybrid Method 
through the systematic exclusion of medical records from review, which was unrelated to the rate 
of medical records received for validation. Blue Advantage Plus did not calculate the measure. 
Finally, it was noted that although MC+ MCOs have the option of calculating this measure through 
either Hybrid or Administrative Method, the one MC+ MCO (Health Care USA) that calculated the 
measure via Administrative Method had a valid, but very low rate for the measure. The Adolescent 
Immunization Status, Combination #| measure requires data from as early as 1990, before the 


implementation of the MC+ Managed Care Program. As such, administrative data alone may not 
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capture all the immunizations necessary to calculate a hit. Given the variation in the rate of 
calculation (e.g., incorporating administrative data) and the impact of compliance with specifications 
(e.g., including all medical records for the Hybrid Method of calculation) across MC+ MCOs as well 
as the low rate of validation of measures, this measure does not provide a valid index of 


performance of the MC+ Managed Care Program. 


The low rates of adolescent immunizations may be accounted for by a number of factors. First, the 
measurement of this rate is retrospective back to 1990, prior to the implementation of the MC+ 
Managed Care Program in Missouri. Although members who were I|3 years of age were enrolled in 
an MC+ MCO for one year continuously to qualify for inclusion in this measure, there is no MC+ 
MCO administrative history for immunizations dating back to birth (e.g., Hepatitis B). Second, the 
ability to retrieve medical records of immunizations from 1990 is complicated by factors such as 
member (and possibly provider) mobility from 1990 to 2003. Third, the documentation of 
immunizations is problematic, as they may not actually be recorded in the member’s medical record 
but may have been administered by a school nurse, a local health initiative, or by a Local Public 
Health Agency (LPHA). This is the reason that MC+ MCOs are encouraged to include data from 
the State Public Health Immunization Registry (MOHSAIC). 


Another limitation to the calculation of valid rates for Adolescent Immunization Status, Combination 
#| is the level of completeness of the State Public Health Immunization Registry (MOHSAIC). 
Although most LPHAs (98% in Missouri in 1992!2) use the Registry, it is not clear how frequently 
the Registry is used to record immunizations. In addition, few private providers maintain the 
resources necessary to enter immunizations into the Registry (9% participation in Missouri in 
1992!3). The Centers for Disease Control (CDC) estimate that the proportion of children under 6 
year of age with two or more immunizations documented in the Registry in Missouri (1992) was 
65%!4, but there are no similar estimates of the rate of use for adolescents immunizations.'5 Schools 
and non-profit organizations also administer immunizations, but may not frequently use the Registry 
to record immunizations. During the course of examining MC+ MCO extract files, approximately 


8% to 25% of administrative hits for immunizations were captured from the MOHSAIC system. 


2 Centers for Disease Control Immunization Registry Annual Report CY2002. Represents the program of 
provider sites reporting to the registry within the last six months of 2002. 

‘3 Ibid. 

™ Ibid. 

*° Personal Communication with Angela Salazar, Public Health Analyst, Immunization Registry Support Branch, 
Immunization Services Division, National |mmunization Program, Centers for Disease Control and Prevention, 
April 11, 2005. 
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Although the level of completeness of the Registry is unknown, it does serve as a useful source for 
contributing to a more complete accounting of immunizations for the calculation of this measure. 
Another possible limitation of calculating the rate is that the Vaccines For Children (VCF) program 
is not popular among private providers. Although MC+ MCOQOs are required to have all providers 
signed on with the VFC Program, some providers do not participate. This may result in more 
referrals to LPHAs for immunizations, which are not billed to the MC+ MCO or otherwise captured 
as having been completed. The Hepatitis B measure is also difficult to track administratively, as the 
claim may come in as a pharmacy code and not a CPT code. Finally, the Family Rights and Privacy 
Act (FRPA) is a barrier to MC+ MCOs for directly obtaining immunization histories from schools 


without individual parental consent. 


Four MC+ MCOs were Substantially or Fully Compliant with the specifications for calculating the 
HEDIS 2004 Adolescent Well-Care Visits measure. One MC+ MCO (Community Care Plus) used 
2002 events for calculating the measure; one MC+ MCO (Mercy Health Plan) did not follow the 
specifications for the Hybrid Method; and one MC+ MCO (Missouri Care) had a low rate of medical 
records despite of good rate of submission. This was likely related to the ability to identify the 
rendering providers at the time of EQRO requests for medical records due to personnel turnover 
since the time those records were collected for calculation. The statewide rate for the Adolescent 
Well-Care Visits measure does not provide a valid index of performance of the MC+ Managed Care 


Program. 


The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was calculated 
according to specifications by all MC+ MCOS and represents a valid index of performance of the 


MC+ Managed Care Program for MC+ MCO comparisons. 


Conclusions 


STRENGTHS 


|. AllMC+MCOs were Fully Compliant with the specifications for calculation of the HEDIS 2004 
Use of Appropriate Medications for People with Asthma measure which represents a valid 
measure of performance of the MC+ Managed Care Program. 


2. The rate for the HEDIS 2004 Use of Appropriate Medications for People with Asthma measure 
across all MC+ MCOs approached the National Commercial and the National Medicaid rates. 


3. Two MC+ MCOs (HealthCare USA and Family Health Partners) were Fully Compliant with 
specifications for the HEDIS 2004 Adolescent Immunization Status Combination #I measure. 
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Family Health Partners was Substantially Compliant with the calculation of the Adolescent 
Immunization Status, Combination # | measure, and the rate exceeded the National Medicaid 
and National Commercial rates for this measure. 


Four MC+ MCOs (HealthCare USA, Family Health Partners, FirstGuard, Blue Advantage Plus) 
were Substantially or Fully Compliant with the specifications for calculating the HEDIS 2004 
Adolescent Well-Care Visits Measure. 


HealthCare USA and FirstGuard rates for the HEDIS 2004 Adolescent Well-Care Visits 
measure exceeded the National Commercial rate; and HealthCare USAs combined rate also 
exceeded the National Medicaid rate. 


In calculating the measures, MC+ MCOs have adequate management information systems for 
capturing and storing enrollment, eligibility, and claims information for the calculation of the 
three HEDIS 2004 measures validated. 


There was good integration of multiple data sources for rate calculation. 


Among MC+ MCOs, there was generally good documentation of the HEDIS 2004 rate 
production process. 


. Four of the MC+ MCOs (Mercy Health Plan, HealthCare USA, FirstGuard, Blue Advantage Plus) 


used NCQA-certified software for sampling, validating data, programming specifications, and 
calculating the HEDIS 2004 measures. This software is required to pass a process using test files 
developed by NCQA, assuring that specific standards are Met for the valid calculation of 
measures. 


. Several MC+ MCOs (HealthCare USA, Missouri Care, Blue Advantage Plus, and Community 


Care Plus) responded to the preliminary findings of the performance measure validation by 
identifying corrective actions plans for the next reporting cycle. 


AREAS FOR IMPROVEMENT 


The HEDIS 2004 Adolescent Immunization Status, Combination #| measure was unable to be 
validated for five of the seven MC+ MCOs and does not represent a valid measure of 
performance of the MC+ Managed Care Program. 


For one MC+ MCO (HealthCare USA), the Adolescent Immunization Status Combination #| 
rate validated was very low. HealthCare USA was the only MC+ MCO not using the Hybrid 
Method for calculating the measure, which may capture more immunizations given that 
retrospective information prior to the implementation of the MC+ Managed Care Program is 
required. HealthCare USA plans to use the Hybrid Method for calculation of this measure in 
the future. 


The HEDIS 2004 Adolescent Immunization Status, Combination #| measure was unable to be 
validated for three of the seven MC+ MCOs and does not represent a valid measure of 
performance for the MC+ Managed Care Program. 


For the validation of the two measures using the Hybrid Method of calculation (HEDIS 2004 
Adolescent Immunization Status, Combination #I and Adolescent Well-Care Visits), the rate of 
submission of medical records for the sample of medical records (fewer than 30 for each MC+ 
MCO) was a likely contributor to the error rate (Community Care Plus, Missouri Care, 
FirstGuard). 
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5. One MC+ MCO (Mercy Health Plan) did not follow the Hybrid Method specifications for the 
calculation of the HEDIS 2004 Adolescent Immunization Status, Combination #I and Adolescent 
Well-Care Visits measure, resulting in ratings of Not Valid for the measures. 


6. One of the three MC+ MCOs (Community Care Plus) that used software that was not certified 
by NCQA for the calculation of the HEDIS 2004 measures used the incorrect measurement 
year (2002) for the calculation of the Adolescent Well-Care Visits measure, rendering the rate 
Not Valid. Automated and manual edit checks should detect such errors. Upon learning of the 
error from the EQRO, Community Care Plus re-submitted the rate using the correct 
measurement year (2003) to the SPHA. 


7. MC+MCO personnel had difficulty identifying the appropriate personnel/vendors and the 
requested information and data required for validation of the measures, including 
documentation (medical records and data sheets) and data (numerators, denominators, and 
specified fields) for the validation. It appeared that the process of using multiple vendors for 
medical record reviews, claims administration, and rate calculation presented a challenge for the 
MC+ MCOs in fully understanding the performance measure rate calculation process. Given 
that this was the first year for performance measure validation, it is anticipated that the process 
of validation will provide context for understanding future data and information requests for the 
validation of performance measures. 


8. Three MC+ MCOs (Missouri Care, Family Health Partners, and FirstGuard) experienced 
turnover in personnel responsible for the calculation of HEDIS 2004 measures, making it difficult 
for the MC+ MCOs to provide the data and information needed to validate the measures. This 
underscores the importance of retaining policies, procedures, and data for valid calculation of 
performance measures from year to year; and providing new staff with training for the 
calculation of performance measures. 


RECOMMENDATIONS 


|. The SMA and SPHA should continue to support efforts to improve the utility and functionality 
of the State Public Health Immunization Registry (MOHSAIC) as well as encourage public, 
private, and non-profit providers of immunizations to use the Registry so as to obtain complete 
information about the level of care provided to MC+ Managed Care Members for the 
administration of immunizations. 


2. For the calculation of the HEDIS Adolescent Immunization Status, Combination #1 measure, the 
Hybrid Method and the incorporation of MOHSAIC data should be required by the SMA to 
facilitate accurate and valid MC+ MCO comparisons and a valid statewide rate for comparison 
of performance with other states. 


3. The SMA should encourage technical assistance regarding the calculation of HEDIS performance 
measures, medical record review processes, and the capture of State Public Health 
Immunization Registry data for the calculation of performance measures. 


4. The SMA should re-validate measures for which all MC+ MCOs were not Fully or Substantially 
Compliant on the calculation of measures. 


5. It is recommended that MC+ MCOs retain medical record review data, records, and electronic 
files used to calculate and report measures or develop procedures for obtaining the small 
samples of medical records for future validation and audit purposes. 
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6. If cost is a factor for MC+ MCOs calculating performance measures using the Hybrid Method in 
compliance with the HEDIS, SMA and SPHA specifications, then the Administrative Method of 
calculation should be used. 


7. Ensure that MC+ MCOs understand the need to calculate and report performance measures to 
the SMA and SPHA despite the NCQA requirements and schedule. Given that these measures 
will not always be audited by NCQA auditors due to their rotation schedule, they may be good 
measures to audit in the future. 


8. MC+ MCOQOs with significantly lower rates of eligible members and administrative hits should 
closely examine the potential reasons for fewer members or claims identified. This may be due 
to member characteristics, but is more likely due to claims administration procedures and 
system characteristics such as the proportion of members receiving services from capitated 
providers. Identifying methods of improving administrative hits will improve the accuracy in 
calculating the measures. 


9. MC+MCOQOs should focus on improving the validity of all claims data to improve the validity of 
the performance measure calculation process. Specific recommendations for each MC+ MCO 
are provided in the individual MC+ MCO summaries. 


10. MC+ MCOs using the Hybrid Method of calculation should establish and document procedures 
for medical record reviews consistent with the performance measure specifications and carefully 
train medical record reviewers, providing written review manuals and procedures. 


Il. MC+ MCOs should target the improvement of provider documentation of medical histories and 
anticipatory guidance to improve the rate of Adolescent Well-Care Visits. 


12, MC+ MCOs need to assess the statistical significance of changes in performance measures over 
time so that possible attributions can be made regarding stability or change from year to year. 


13. The SMA will need to consider the validation of other performance measures that are used for 
program monitoring and evaluation. One measure set that was developed by the Missouri 
Department of Mental Health and MC+ MCOs is the Mental Health Indicator set, which is used 
by the MC+ Managed Care Program Mental Health Subcommittee. Previous EQRO review of 
raw data and comparison of MC+ MCO performance has raised questions about the reporting 
and calculation of the measures. The measures were calculated by the Missouri Department of 
Mental Health and have recently been calculated and reported by New Directions, one of the 
Behavioral Health Organizations for MC+ MCOs. Two measures for which there was concern 
about the proper calculation were the 7-day Follow-Up and 30-day Follow-Up After Psychiatric 
Hospitalization. Although the Mental Health Indicators are based on HEDIS criteria, they do not 
specifically follow all criteria for calculation. These two measures are also HEDIS measures, 
with very prescribed specifications for calculation. It was reported that some MC+ MCOs 
report their HEDIS-calculated rates to New Directions, while others provide the data according 
to the criteria for reporting as a Mental Health Indicator. This may result in non-comparability 
of the measures across MC+ MCOs as reported in the Mental Health Indicator set. If these 
measures are required by the SMA, it is recommended that they be calculated using one set of 
specifications. Since MC+ MCOs are likely to be more familiar with the HEDIS specifications for 
the calculation of the measures, it is recommended that the Mental Health Subcommittee accept 
and report this method of calculation for the two measures. If there is a compelling reason to 
use the Mental Health Indicator set definitions, then MC+ MCOs should be all required to 
follow the same specifications for calculation. The SMA plans to continue to require the 
calculation of the 7-day Follow-Up and 30-day Follow-Up After Psychiatric Hospitalization 
measure for mental health services although it is being rotated by HEDIS. Given the importance 
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of the two measures, it is recommended that the calculation of either the HEDIS or Mental 
Health Indicator measures of the 7-day and 30-day Follow-Up After Psychiatric Hospitalization 
measures be validated by the EQRO. Given the complexity of the calculation of immunization 
measures, another measure recommended for validation by the EQRO is the HEDIS Childhood 
Immunization measure. 


14. Finally, it is recommended that a qualified external entity be used to calculate and report 
performance measures comparing MC+ MCOs. 
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SECTION THREE: VALIDATION OF ENCOUNTER DATA 





Definition 


‘For the purposes of this protocol, an encounter refers to the electronic record of a service 
provided to an MCO enrollee by both institutional and practitioner providers (regardless of how 
the provider was paid) when the service would traditionally be a billable service under Fee-for- 


Service (FFS) reimbursement systems.”’!é 


An encounter is defined as the unit of service provided to a Member by the MCO. Encounter data 
provides the same type of information found on a claim form. It does not substitute for medical 
record documentation, but should be consistent with and supported by medical record 
documentation (e.g. date of procedure, type of procedure). The MC+ MCOs’ contract with the 
State Medicaid Agency (SMA; Missouri Department of Social Services, Division of Medical Services; 
DMS) details the requirements for an acceptable submission of an encounter. The SMA’s 
requirements for encounter data submitted by the MC+ MCOs include the type of encounter data 


and required data fields. 


Purpose and Objectives 


“Encounter data can be used to assess and improve quality, as well as monitor program integrity and determine 
capitation payment rates. However, in order for encounter data to effectively serve these purposes, it must be 
valid; i.e., complete and accurate... This protocol specifies processes for assessing the completeness and 
accuracy of encounter data submitted by MCOs and PIHPs to the State. It also can assist in the improvement 
of the processes associated with the collection and submission of encounter data to State Medicaid agencies.”!7 


Three objectives for the encounter validation were identified. They included assessing the quality of 
data for required fields for each claim type, evaluating the representativeness (or completeness) of 
the SMA encounter claims database for MC+ MCO paid and unpaid claims, and validating medical 
records against the SMA encounter claims database. The following were the objectives and 


associated evaluation questions. 


‘© Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Encounter Data: A protocol for use in Conducting Medicaid External Quality Review Activities, Final Protocol, 
Version 1.0, May 1, 2002. Washington, D.C.: Author. 


1” Ibid. 
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|. The first objective was to obtain a baseline of the SMA encounter claim database quality 
(completeness, accuracy, and reasonableness). The alternative hypothesis was that all data fields 
in the SMA encounter claims database consist of valid (complete, accurate, and reasonable) 
encounter claim data. Appendix 5 shows the recommended minimum criteria established for 
completeness and accuracy of specific data fields. Several evaluation questions were addressed: 


e What is the baseline level of completeness, accuracy, and reasonableness of the critical fields? 
e What is the level of volume and consistency of services? 

e What are the data quality issues associated with the processing of encounter data? 

e What problems are there with how files are compiled and submitted by the MCO? 

e What types of encounter claim data are missing and why? 


2. The second objective was to validate the SMA encounter claims (paid) database against medical 
record documentation and obtain a baseline fault (error) rate for the level of accuracy of the 
SMA encounter claims database relative to the services delivered by MC+ MCO providers. The 
alternative hypothesis was that there is a 100% match between the encounter claim data in the 
medical record and the data in the SMA encounter claims database. Accuracy or match rates of 
70% or greater are anticipated for new Medicaid managed care organizations!8, Several 
evaluation questions were addressed: 


e To what extent do the claims in the SMA encounter claims database reflect the information documented in 
the medical record? 

e What is the fault/match rate between SMA encounter claims and medical records? 

e What types of errors are noted? 


3. The third objective was to examine the match between MC+ MCO claims (paid and unpaid) and 
the SMA encounter paid claims database. This would facilitate identification of the level of 
completeness of the SMA encounter claims database as represented by MC+ MCOs paid claims. 
The alternative hypotheses were that 100% of MC+ MCO paid claims are represented in the 
SMA encounter claims database, and 0.00% of MC+ MCO unpaid claims are represented in the 
SMA encounter claims database. Several evaluation questions were posed: 


e What types of paid encounter data are missing and why? 

e What is the fault/match rate of paid and unpaid encounter claims in the SMA encounter claim database and 
the MC+ MCO claims database? 

e What services are being provided that are not being paid? 

e How many services are being provided that are not being paid? 


18 Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 
Second Edition. 
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Technical Methods 


TIME FRAME 
The time frame of dates of service from January |, 2004 through March 31, 2004 was selected by 


the SMA for the three encounter data validation objectives. 


PROCEDURES FOR DATA COLLECTION 

For the first objective, the SMA encounter claims extract file was used to examine the 
completeness, accuracy, and reasonableness of the critical fields; and to calculate the rate of each 
claim type per 1,000 members by MC+ MCOs. There are six claim types described in the SMA 
Health Plan Layout Manual: | = Inpatient claim type; M = Medical claim type; O = Outpatient 
Hospital claim type; D = Dental claim type; H = Home Health claim type; and P = Pharmacy claim 
type. Outpatient and Home Health claim types are submitted using a Universal Billing (UB-92) file 
layout, Medical and Dental claim types are submitted using a National Standard Format/Centers for 
Medicare and Medicaid Services 1500 (NSF/CMS 1500) file layout, and the Pharmacy claims are 
submitted using the National Council for Prescription Drug Programs, version 3 file layout (NCPDP 
v.3.0). All claims are sent from the MC+ MCOs to the SMA through the SMA claims vendor, 
InfoCrossing (recently renamed from Verizon), and claim types are assigned by the Medicaid 


Management Information System (MMIS). 


After review and approval of the technical methods and objectives by the SMA, the EQRO 
reviewed, discussed with the SMA, and submitted a data request (see Appendix 5) for the SMA 
encounter claims extract file to be validated for each claim type and each MC+ MCO. The file 
request was made to the SMA on October |6, 2004 and received on November 18, 2004 by the 
EQRO, with additional files provided through January 25, 2004. The SMA reviewed and approved 


the data request and parameters for the designated the fields to be validated by the EQRO. 


The comma-delimited text file was imported into the Statistical Package for Social Sciences, version 
[2 (SPSS v.12). Upon review of the imported file, the EQRO promptly notified the SMA of 
misaligned data fields at the end of the encounter claims extract file. These fields were: 


OUTPAT-DTL-PROC-MOD-P 
OUTPAT-DTL-PROC-MOD.-| 
OUTPAT-PERV-PROV-MCARE-NUM 
OUTPAT-DTL-DIAG-CODE 
OUTPAT-DTL-PAID-AMT 
OUTPAT-DTL-EOB-CODE 
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The SMA noted the fields were not in error, and the two procedure modifier fields were combined 
into one field. The EQRO attempted to parse these fields. The SMA Health Plan Layout Manual 
specifies two 2-digit modifier fields. However, the data in the merged procedure modifier fields 
consisted of either no data or a five-digit sequence of data. Subsequent data fields contained varying 
types of information (dollar amounts, alphanumeric characters). The EQRO Project Director and 
three programmers independently attempted to import the original data file into spreadsheet and 
database programs (SPSS v.12, MS Access and MS Excel) with the same results. The SMA was 
notified that the EQRO would need either to omit the fields from the analysis or report the 
procedure code modifier fields as invalid. No additional extract files were provided for analysis and 
the EQRO did not validate the procedure code modifier fields due to concerns about the 


representativeness of these fields!?. 


The number of Medical encounter claims in the SMA encounter claims extract file was used for 
sample size estimation for the second objective and analysis of the evaluation questions. To 
examine the degree of match between the SMA encounter claims database and medical record 
procedures and diagnoses, 100 encounters from each MC+ MCO were randomly selected from 
Medical claim types for the period of January |, 2004 through March 31, 2004 for medical record 
review. Appendix 6 contains letters of request to providers for medical records, the Table of 
Contents for the Medical Record Review Training Manual, and copies of medical record review 
tools. Several challenges in requesting the data were addressed. The SMA database does not 
include the names and addresses of the professional providers delivering the services. Rather, the 
MC+ MCO is recorded as the provider. This necessitated sending to MC+ MCOs listings of the 
encounter claim data elements provided to the EQRO by the SMA for the validation so that 
provider names and addresses could be used to send the medical record requests. MC+ MCOs 
returned over |,139 provider names and addresses for the /00 selected encounters. Due to time 
constraints, the EQRO mailed requests to all providers. However, only 700 encounters would 
expect to have valid matches and the goal was to exclude from analysis any records that did not 
match and should not have matched the SMA encounter claims sampled. A second request was sent 
to MC+ MCOs with additional information on each encounter (including the procedure code and 
diagnosis code) to identify the exact provider associated with the encounter claims of interest. 


Nevertheless, MC+ MCOs identified more than one provider for each encounter (see Appendix 5). 


19 Personal communications with J udy Muck, Assistant Deputy Director of MC+ Managed Care, January 22, 
2005 and January 26, 2005. 
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It was subsequently discovered that the MC+ MCQOs retain SMA assigned Individual Control 
Numbers (ICNs) to identify unique encounter claims. For those records in which the provider 
associated with the claim were in question, the SMA provided the ICNs which were forwarded to 
MC+ MCOs to identify the exact provider associated with the claim. In some cases, additional 
providers not included in the original request were added. During the months that specific 
providers were being identified, all medical records were reviewed. Only the findings of those 
medical record reviews for which the provider could be confidently associated with the encounter 
claim were included in the final analysis. Another challenge identified in the present process was the 
ability of MC+ MCOs to identify unique providers. In many instances, the provider was listed as a 
hospital system. In addition, providers with multiple addresses or the same street address in two 
different cities were found. This made it difficult for the facilities and providers to identify the 


sources and location of medical records. 


For the third objective of comparing the SMA encounter claims with MC+ MCOs’ paid and unpaid 
claims, the SMA encounter claims extract file was parsed by type of file layout (NSF/CMS 1500, UB- 
92, or NCPDP v.3.0) in preparation for matching against MC+ MCO paid and unpaid claims. A 
cross-walk for matching MMIS field names with those of the three national standards file layouts was 
developed and submitted to the SMA for review (February 8, 2005) and approval (March 29, 2005). 
MC+ MCOs were requested to provide paid and unpaid claims for the designated time frame on the 
sample of members selected by the EQRO, using the file layout required by the SMA prior to 
October, 2003. For a variety of reasons, MC+ MCOs were unable to consistently provide paid and 
unpaid claims and/or the requested file layouts. This precluded the planned analyses, but 
observations regarding the file submissions were provided in the MC+ MCO summaries for data 


quality improvement efforts. 


ANALYSES 
To assess the accuracy and completeness of the SMA encounter claims database, the SMA 
encounter claims extract file for all MC+ MCO paid encounter claims representing services 


rendered from January |, 2004 through March 31, 2004 was analyzed for completeness, accuracy, 


|’ 


and reasonableness (validity) of the data in each “critical”, or required field examined. The Inpatient, 


Medical, Dental, Home Health, Outpatient Hospital, and critical fields were chosen by the SMA for 


analysis, with an established threshold of 100% for completion, accuracy, and validity: 
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Medical (NSF/ CMS 1500) Claim Type Inpatient (UB-92) Claim Type 
Outpatient Claim Type Inpatient Claim Type 
Recipient 1D Recipient 1D 
First Date of Service Admission Type 
Last Date of Service Admission Date 
Place of Service Discharge Date 
Units of Service Bill Type 
Procedure Code Patient Discharge Status 
Inpatient Diagnosis (five diagnosis fields) Inpatient Diagnosis (five diagnosis fields) 
First Date of Billing 
Dental (NSF/ CMS 1500) Claim Type Last Date of Billing 
Outpatient Claim Type Revenue Code 
Recipient |D Units of Service 
First Date of Service 
Last Date of Service Outpatient Hospital (UB-92) Claim Type 
Units of Service Outpatient Claim Type 
Procedure Code Recipient |D 
First Date of Service 
Home Health (UB-92) Claim Type Last Date of Service 
Outpatient Claim Type Place of Service 
Recipient |D Units of Service 
First Date of Service Procedure Code 
Last Date of Service Inpatient Diagnosis (five diagnosis fields) 
Units of Service 
Procedure Code Pharmacy (NCPDP v.3.0) 
Revenue Code Recipient |D 
Inpatient Diagnosis (five diagnosis fields) Dispensing Date 


Pharmacy Prescription Number 
Drug Quantity Dispensed 
Number of Days Supply 
National Drug Code 


Each field was examined for the presence or absence of data (completeness), the correct type and 
size of information (accuracy), and the presence of valid values (reasonableness) or validity using the 
criteria listed below. Appendix 5 contains the parameters for the validation of encounter claims 
fields for each of the six encounter claim types, the procedure codes, and the diagnosis codes. 


Appendix 5 also shows the recommended threshold for validity of specific data fields. 


Completeness: The extent to which an encounter claim field contains data (either 
present or absent). 


Accuracy: The extent to which an encounter claim field contains the correct 
type of information (e.g., numeric, alpha, alphanumeric) in the 
proper format (e.g., mm/dd/yyyy for date field). 


Reasonableness (Validity): The extent to which an encounter claim field represents a valid value 
(e.g., an actual procedure code, actual birth date) 





For the validation of the SMA encounter claims extract file with MC+ MCO medical records, the 
goal was to validate the procedure code and diagnosis code fields in the SMA encounter claims 
database against the information provided in the medical record. The minimum number of records 


required for the evaluation of two variables (procedure and diagnosis) with an estimated error rate 
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of 30% (based on Medstat estimates2°), reliability of 1.96 (95% statistical significance), and a 
meaningful difference of 55% were calculated using the number of Medical encounters in the SMA 
encounter claims file for each MC+ MCO (see Equation |). There were no differences in the 
number of required records for MC+ MCOs, with the minimum required sample size of 88. A total 
of 100 encounters for each MC+ MCO were randomly selected for medical record review using a 
probability sample. The results of matches between the medical record and encounter claims data 
for the diagnosis code and diagnosis description across both the medical record and claim history 
were combined to calculate the number of matches for diagnosis. Similarly, the results of matches 
between the medical record and encounter claims data for the adequacy of information in the 
medical record as judged by the coder were combined with the number of matches on the claim 
form for the procedure code and description. If there was a match across any of the multiple 
variables (diagnosis code and description) or two sources (medical record and claim form) for the 
diagnosis, the record was counted as a “match”. The same approach to calculating matches was 
used for the procedure so that the possible range for matches for each MC+ MCO on each variable 


was 0 to 100, expressed in percents. 


Equation |. Formula for Calculating Minimum Required Sample Size. 

— Zz?7NP,(1-P,) 

(N -1) e?P,2 + 2°P A 1 - P,) 

Where Py, = Estimated True Error Rate; meaningful difference between true and estimated value ; z = level of reliability; 
€=|(Py - meaningful difference)/meaningful difference; N = number of Medicaid Claim Types for the period January |, 
2004-March 31, 2004; n = Minimum required sample size?! 
FINDINGS 
One limitation of the present analysis is that the encounter claim completeness and accuracy analysis 
was based on paid encounter claims and did not account for all claims that were submitted and 
rejected through system edits. Another limitation was that there were some discrepancies noted 
between the payable procedure code list and the SMA-identified parameters for the procedure code 
field. The payable procedure code list contained values ranging from three to five digits, while the 
parameters identified by the SMA for the present validation process indicated valid lengths of data of 
five digits. The source of the procedure code file was an internal database, which comes directly 


from of the MMIS system. The source of the diagnosis code list was an ad hoc report. The ad hoc 


20 Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 
Second Edition. 

*1 Levy, P.S. & Lemeshow, S. L. (1999). Sampling of Populations: Methods and Applications, Third Edition, 
John Wiley and Sons: New York; see box 3.5 for Exact and approximate sample sizes required under simple 
random sampling for proportions. 
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data were taken directly from the MMIS system.22 Finally, because the SMA encounter claims extract 
file was for service dates from January |, 2004 through March 31, 2004 and some service dates 
might extend beyond this period if the first date of service was later in the time frame (e.g., March 
31, 2004), the last date of service extended beyond the time frame specified by SMA parameters for 
the validation process (e.g., a Discharge Date of April |, 2004). When last dates of service appeared 
to be within a reasonable time frame, dates outside the valid range were not interpreted and are 
presented for informational purposes only. In addition, the second through fifth diagnosis code 
fields are required when the information is available. Not all encounters had five diagnoses. 
Therefore, 100.00% completion of these fields would not be expected. Conclusions regarding the 
extent to which the encounter claims database reflects the accuracy and completeness of rejected 
claims cannot be drawn. Data are presented in the aggregate and are available at the MC+ MCO 
level in the individual MC+ MCO summaries. The findings of the encounter data validation are 
presented in response to each evaluation question, by claim type and critical field for all MC+ 


MCOs. 


22 Personal communication with Shelley Farris, December 02, 2004 re: Diagnosis, Procedure code 
documentation question and follow-up. 
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What is the Baseline Level of Completeness, Accuracy, and Reasonableness of 
the Critical fields? What Types of Encounter Claim Data are Missing and Why? 


For the Medical claim type, there were a total of 999,140 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004 (see Table 37). 


The Outpatient Claim Type field was 100.00% complete, accurate and valid. 


2. The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

3. The Outpatient First Date of Service field was 100.00% complete and accurate, and 99.95% valid. Invalid dates of 
service ranged from 01/05/2001 — 12/16/2003. 

4. The Outpatient Last Date of Service field was 100.00% complete and accurate, and 99.88% valid. Invalid dates of 
service ranged from 04/01/2004 — 08/21/2004. 

5. The Outpatient Units of Service field was 100.00% complete and accurate. Valid values of the appropriate type 
were present 99.93% of the time. The Units of Service field must contain five numeric characters representing 
values from “00001” — “99999”. Family Health Partners had two values of “00000” and Community Care Plus 
had 679 values of “00000”. 

6. The Outpatient Procedure Code field was 100.00% complete and accurate, and 97.36% valid. Invalid procedure 
codes included the W0025 codes. Please refer to individual MC+ MCO reports on specific invalid procedure 
codes found in this field. 

7. The Outpatient Place of Service field was |00.00% complete, accurate and valid. 

8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 

9. Although the second through fifth Diagnosis Code fields are optional according to the Health Plan Record 
Layout Manual, they fell well below the 100% threshold established by the SMA for this validation. The second 
Diagnosis Code field was 34.49% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The third Diagnosis Code field was 16.02% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 7.46% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fifth Diagnosis Code field was .78% complete, accurate and valid. The remaining 
fields were blank (incomplete, inaccurate, and invalid). 
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Table 37. Encounter Data Validation of Critical Fields, All MC+MCOs, Medical Claim Type. 


Ol dis(et-] me) mae) Ma (-1[e| 
Outpatient Claim Type [OUTPAT-CLAIM-TYPE] 
Recipient 1D [OUTPAT-PROCESSED-RECIP-ID] 
First Date of Service [OUTPAT-FIRST-DT-SVC] 
Last Date of Service [OUTPAT-LAST-DT-SVC] 
Units of Service [OUTPAT-UNITS-SVC] 
Outpatient Procedure Code [OUTPAT-DTL-PROC] 
Outpatient Place of Service [OUTPAT-PLACE-OF-SVC] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 


_ Total Claims 


999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
344,595 
160,104 
74,577 
7,833 


999,140 


MabielanitcheleleM aa--i sels 


a 0 ee ee ee ee eee ee eee 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
34.49% 
16.02% 
7.46% 
0.78% 


Note: Based on state extract file of dates of service from January |, 2004- March 31, 2004. 
Source: Missouri Department of Social Services, Division of Medical Services. 
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Correct Size 


999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
999,140 
344,595 
160,104 

74,577 


7,833 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
34.49% 
16.02% 
7.46% 


0.78% 


Correct Type of 
Mahcelauit-haceyal 


999,140 
999,140 
999,140 
999,140 
998,459 
999,140 
999,140 
999,140 
344,595 
160,104 

74,577 


7,833 


100.00% 
100.00% 
100.00% 
100.00% 
99.93% 
100.00% 
100.00% 
100.00% 
34.49% 
16.02% 
7.46% 


0.78% 


Valid Value 


999,140 
999,140 
998,647 
997,958 
998,459 
972,805 
999,140 
999,140 
344,595 
160,104 

74,577 


7,833 


% 
100.00% 
100.00% 

99.95% 
99.88% 
99.93% 
97.36% 
100.00% 
100.00% 
34.49% 
16.02% 
7.46% 


0.78% 
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For the Dental claim type, there were a total of 125,236 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 
accurate and valid for all MC+ MCOs. 


For the Home Health claim type, there were a total of 828 encounter claims paid by the SMA for 
the period January |, 2004 through March 31, 2004. 


For the Inpatient claim type, there were a total of | 1,619 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004 (see Table 38). 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 


2. The Recipient ID field was 100.00% complete, accurate and valid. 

3. The Admission Type field was 100.00% complete, accurate and valid. 

4. The Admission Date field was 100.00% complete and accurate; and 95.12% valid. Invalid dates ranged from 
01/22/2003 — 12/31/2003. 

5. The Discharge Date field was 100.00% complete with the correct number of characters (size). The correct type 
of information (date format) was present 98.35% (with entries of “99999999” by all MCOs, with the exception 
of Family Health Partners). Valid values were present 94.35% of the time. In addition to the invalid “99999999” 
entries, invalid dates ranged from 04/01/2004 — 07/29/2004. 

6. The Bill Type field was 100.00% complete, accurate and valid. 

7. The Patient Status field was 100.00% complete with the correct number of characters (size). This field requires 
two digits. Missouri Care, FirstGuard and HealthCare USA had two-digit entries of “16”, “63”, “35”, and “24”, 
which were the correct type, but were invalid values. 

8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 

9. The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for completeness, 
accuracy, and validity established by the SMA (45.16%, 35.71%, 13.79%, and 7.81%, respectively). The remaining 
fields were blank (incomplete, inaccurate, and invalid). 

lO. The First Date of Service field was 100.00% complete and accurate, and 95.84% valid. Invalid dates of service 
ranged from 01/22/2003 — 12/31/2003. 

Il. The Last Date of Service field was 100.00% complete and accurate, and 52.68% valid. Invalid dates of service 
ranged from 04/01/2004 — 07/29/2004. 

|2. The Revenue Code field was 99.98% complete, accurate, and valid. Invalid fields were either blank or contained 
an invalid entry of “080”. 

13. The Units of Service field was 100.00% complete, 99.76% accurate and 99.75% valid. Invalid values were 
primarily accounted for by invalid entries of “00000” from Missouri Care and HealthCare USA. 
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Table 38. Encounter Data Validation of Critical Fields, All MC+MCOs, Inpatient Claim Type. 


Critical or Fatal Field 
Inpatient Claim Type [INPAT-CLAIM-TYPE] 
Recipient |D [INPAT-RECIP-!ID] 
Admission Type [INPAT-ADMIT-TYPE] 
Admission Date [INPAT-ADMIT-DT] 
Discharge Date [INPAT-MED-DSCHG- DT] 
Inpatient Bill Type [INPAT-BILL-TYPE] 
Patient Status [INPAT-PATIENT-STAT] 


Diagnosis [INPAT- DX] 
Diagnosis [INPAT-DX] 
Diagnosis [INPAT- DX] 
Diagnosis [INPAT-DX] 
Diagnosis [INPAT- DX] 


First Date of Service [INPAT-FIRST-DT-SVC] 
Last Date of Service [INPAT-LAST-DT-SVC] 
Revenue Code [INPAT-REVENUE-CD] 

Units of Service [INPAT-UNITS-SV] 


Total Claims 


Information Present 


# 
111,619 
111,619 
111,619 
111,619 
111,619 
111,619 
111,619 
111,619 

50,407 
39,858 
15,390 

8,/13 
111,619 
111,619 
111,601 
111,619 


% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 

45.16% 
35.71% 
13.79% 
7.81% 
100.00% 
100.00% 
99.98% 
100.00% 


Correct Size 


# 
111,619 
iol? 
111,619 
111,619 
111,619 
iol? 
111,619 
111,619 

50,407 
39,858 
15,390 

8,/13 
111,619 
ii ole 
111,601 
111,619 


Note: Based on state extract file of dates of service from January |, 2004- March 31, 2004. 


Source: Missouri Department of Social Services, Division of Medical Services. 
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% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 

45.16% 
35.71% 
13.79% 
7.81% 
100.00% 
100.00% 
99.98% 
100.00% 


Correct Type of 
Mavceleertchacela 


# 
111,619 
111,619 
111,619 
111,619 
109,772 
111,619 
111,543 
111,619 

50,407 
39,858 
15,390 

8,/13 
111,619 
111,619 
111,600 
111,344 


% 
100.00% 
100.00% 
100.00% 
100.00% 

98.35% 
100.00% 
99.93% 
100.00% 
45.16% 
35.71% 
13.79% 
7.81% 
100.00% 
100.00% 
99.98% 
99.75% 


WE-VICe YET LU(= 


# 
111,619 
111,619 
111,619 
106,173 
105,315 
111,619 
111,543 
111,619 

50,407 
39,858 
15,376 

8,/13 
106,972 
106,544 
111,600 
111,343 


% 
100.00% 
100.00% 
100.00% 

95.12% 
94.35% 
100.00% 
99.93% 
100.00% 
45.16% 
35.71% 
13.78% 
7.81% 
95.84% 
95.45% 
99.98% 
99.75% 
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For the Outpatient Hospital claim type, there were a total of 489,432 encounter claims paid by the 
SMA for the period January |, 2004 through March 31, 2004 (see Table 39). 


|. Two of the seven MC+ MCOs (Missouri Care and Mercy Health Plan) had 100.00% complete, accurate and valid 
data for all fields examined. 


The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete and accurate, and valid. 
The Last Date of Service field was 100.00% complete and accurate, and valid. 
The Units of Service field was 100.00% complete, accurate and valid. 


The Outpatient Procedure Code field was |00.00% complete, 94.77% accurate and 92.75% valid. A large 
proportion of the invalid codes were accounted for by codes ranging from “0220” — “0983” submitted by 
FirstGuard. This field requires five alphanumeric characters. 


=e oe Se YS PY 


8. The Outpatient Revenue Code field was 100.00% complete, 83.25% accurate and 83.25% valid. Invalid codes 
were primarily accounted for by invalid entries of “O00” from HealthCare USA and Community Care Plus. 


9. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


10. Although the second through fifth Diagnosis Code fields are optional according to the Health Plan Record 
Layout Manual, they fell well below the 100% threshold established by the SMA for this validation. The second 
Diagnosis Code field was 30.39% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The third Diagnosis Code field was 13.98% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 4.56% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fifth Diagnosis Code field was |.33% complete, accurate and valid. The remaining 
fields were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were a total of 737,511 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, accurate and 
valid (Recipient ID, First Date of Service, Prescription Number, Quantity Dispensed, Days Supply, 
and National Drug Code). 
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Table 39. Encounter Data Validation of Critical Fields, All MC+ MCOs, Outpatient Hospital Claim Type. 


Correct Type of 
Information Present Correct Size Mahielaritchacedal Valid Value 
es es er Ss SS 


(Ol gia (er-) me) am at-le-] mg 1-)[e| 





Outpatient Claim Type [OUTPAT-CLAIM-TYPE] 
Recipient 1D [OUTPAT-PROCESSED-RECIP-1ID] 
First Date of Service [OUTPAT-FIRST-DT-SVC] 
Last Date of Service [OUTPAT-LAST-DT-SVC] 
Units of Service [OUTPAT-UNITS-SVC] 
Outpatient Procedure Code [OUTPAT-DTL-PROC] 
Revenue Code [OUTPAT-REVENUE-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 
Diagnosis [OUTPAT-DTL-DIAG-CODE] 

otal Claims 


489,432 
489,432 
489,432 
489,432 
489,432 
489,432 
489,432 
489,428 
148,752 
68,405 
22,299 
ewe) 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
30.39% 
13.98% 
4.56% 
1.33% 


489,432 
489,432 
489,432 
489,432 
489,432 
463,831 
489,393 
489,428 
148,752 
68,405 
22,299 
6,526 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
94.77% 
99.99% 
100.00% 
30.39% 
13.98% 
4.56% 
ifs Si70 


489,432 
489,432 
489,432 
489,432 
489,432 
463,831 
407,457 
489,428 
148,752 
68,405 
22,299 
6,526 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
94.77% 
83.25% 
100.00% 
30.39% 
13.98% 
4.56% 
1.33% 


489,432 
489,432 
489,432 
489,432 
489,432 
453,938 
407,457 
489,428 
148,752 
68,405 
22,299 
67526 


100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
92.75% 
83.25% 
100.00% 
30.39% 
13.98% 
4.56% 
1.33% 


ee ya Oe 





Note: Based on state extract file of dates of service from January |, 2004- March 31, 2004. 


Source: Missouri Department of Social Services, Division of Medical Services. 
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What is the Level of Volume and Consistency of Services? 

One method of examining the level, consistency, and volume of services is to assess the extent to 
which each MC+ MCO is consistent with the remaining MC+ MCOs and the average of all MC+ 
MCOs services represented in the SMA encounter claims database. The level, consistency, and 
volume of services represented in the SMA encounter claims database is a function of the 
acceptance of encounter claim submissions; the process of manipulation of data from national 
standard layouts for Medical (NSF/CMS 1500); Dental (NSF/CMS 1500); Inpatient, Outpatient 
Hospital, Home Health (UB-92); and Pharmacy claims (NCPDP 3.0) into the State MMIS system 
edits; the entry and transmission of data by MC+ MCOs, vendors, and providers; the accessibility of 
services; member utilization patterns; and provider practice patterns. Given that there were no 
baseline data available from the SMA for comparison, analyses are considered exploratory and may 
be used as a baseline for further analysis or interpretation by the SMA and the MC+ MCOs. With 
the large number of members enrolled in each MC+ MCO, it was expected that factors such as 
physician practice patterns and member utilization patterns would not have a statistically significant 
impact on the findings, resulting in all MC+ MCOs having similar rates of encounters per |,000 
members as the rate for all MC+ MCOs. Statistically significant findings are more likely to be a 
function of the data quality and completeness resulting from the processing of data by providers, 


vendors, MC+ MCOs, and the MMIS rather than the accessibility or quality of services. 


Using the SMA encounter claims extract file from January |, 2004 through March 31, 2004, the 
volume of services for each claim type and MC+ MCO was examined. The rate of each claim type, 
regardless of the accuracy, consistency, and validity of the data was examined. The rate of claims 
per 1,000 members based on one quarter of data was calculated by dividing the number of members 
enrolled as of the last week of March 2004, by 4, then calculating the rate of claims per |,000 
members. Figures |3 through [8 illustrate the rates of claim types and the results of two-tailed z- 
tests comparing each MC+ MCO with the statewide rate of claims. Statistically significant 
differences between an MC+ MCO and the rate for all MC+ MCOs at the 95% level of statistical 
significance are indicated by an asterisk. The 95% upper and lower confidence limits are 
represented by the black bars on the y-axis. For comparisons that were statistically significant at the 
95% confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 
significance levels (p < .05) are reported. When there was no statistical significance, the significance 


level is reported as “not significant” (n.s.). 
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Medical encounter claim types consist of claims submitted by providers, vendors, and MC+ MCOs. 
As shown in Figure |3, there was some variability across MC+ MCOs in the statewide rate per 
|,000 members of Medical encounter claim types compared to the rate for all MC+ MCOs 
(9,074.69 Medical encounter claims per 1,000 members). Two MC+ MCOs (HealthCare USA, 
11795.45, z = 1.66 ; 95% Cl: 10707.42, 12883.48; p > .95; and Missouri Care, | 1015.49, z = 1.13; 
95% Cl: 9927.46, 12103.53; p > .95) had significantly higher rates, while two MC+ MCOs 
(Community Care Plus, 8353.31, z = -.69; 95% Cl: 7625.27, 9441.34; p < .05; and Mercy Health 
Plan, 7919.84, z = -.98; 95% Cl: 6831.81, 9007.87; p < .05) had significantly lower rates of Medical 


encounter claims than the rate for all MC+ MCOs. 


Figure 13. Medical Encounters Claim Types per 1,000 Members, January I, 2004 —- March 31, 2004. 
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B® Medical 8,353.31 7,919.84 11,795.45 11,015.49 8,524.16 9,259.3 | 8,671.31 9,074.69 
MC+ MCO 











Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 


Dental encounter claims consist of claims submitted by providers, vendors, and MC+ MCOs. As 
shown in Figure |4, there was some variability across MC+ MCOs in the rate per |,000 members of 
Dental encounter claims compared to the rate for all MC+ MCOs (1137.19 Dental encounter claims 


per 1,000 members). Two MC+ MCOs (HealthCare USA, 1523.63, z = 1.54 ; 95% Cl: 1337.83, 
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1709.42; p > .95; and Family Health Partners, 1362.81, z = .90; 95% Cl: 1177.02, 1548.61; p > .95) 
had significantly higher rates while two MC+ MCOs (FirstGuard, 834.58, z = -1.20; 95% Cl: 648.79, 
1020.38; p < .05; and Blue Advantage Plus, 855.09, z = -1.12; 95% Cl: 669.29, 1040.88; p < .05) had 
significantly lower rates of Dental encounter claims per |,000 members than the rate for all MC+ 


MCOs. 


Figure 14. Dental Encounters per |,000 Members, January I, 2004 —- March 31, 2004. 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per 1,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 
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There were very few Home Health encounter claim types submitted by MC+ MCOs. One MCt+ 
MCO (Blue Advantage Plus, 64.72, z = 2.17; 95% Cl; p > .95) submitted a significantly higher rate of 


Home Health encounter claims than the rate for all MC+ MCOs (7.52; see Figure 15.) 


Figure 15. Home Health Encounter Claim Types per 1,000 Members, January |, 2004 — March 31, 2004. 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 
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Inpatient encounter claim types consist of claims submitted by hospital facilities and MC+ MCOs. 
As shown in Figure |6, there was some variability across MC+ MCOQOs in the rate per 1,000 
members of Inpatient encounter claims compared to the rate for all MC+ MCOs (1013.78 Inpatient 
encounter claims per 1,000 members). Two MC+ MCOs had significantly higher rates of Inpatient 
encounter claims (Mercy Health Plan, | 776.84, z = 1.35; 95% Cl: 1336.27, 2217.41; p > .95; 
HealthCare USA, z = .81; 95% Cl: 1011.27, 1892.42, p > .95). Two MC+ MCOs had significantly 
lower rates of Inpatient encounter claims (Community Care Plus, 237.75, z = -1.23; 95% Cl: 0, 
678.33; p < .05; Blue Advantage Plus, 179.10, z = -1.33; 95% Cl: 0, 619.67; p < .05) than the rate for 
all MC+ MCOs. 


Figure 16. Inpatient Encounter Claim Types per |,000 Members, January I, 2004 —- March 31, 2004. 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 


Outpatient Hospital encounter claim types consist of claims submitted by outpatient hospital 
facilities and MC+ MCOs. As shown in Figure | 7, there was some variability across MC+ MCOs 
compared to the rate for all MC+ MCOs (4528.28 Outpatient Hospital encounter claims per |,000 
members). Two MC+ MCOs (Missouri Care, 5985.78, z = .79; 95% Cl: 4691.21, 7280.34; p > .95 
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and Family Health Partners, 6591.82, z = 1.14; 95% Cl: 5297.25, 7886.38; p > .95) had significantly 
higher rates of Outpatient Hospital encounter claims, while one MC+ MCO had significantly lower 
rates of Outpatient Hospital encounter claims per |,000 members (Community Care Plus, z = -1.79; 
95% Cl: 179.15, 2768.28; p < .05) than the rate for all MC+ MCOs. 


Figure 17. Outpatient Hospital Encounter Claim Types per 1,000 Members, January |, 2004 —- March 31, 
2004. 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 
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Pharmacy encounter claim types consist of claims submitted by pharmacy providers and MC+ 
MCOs. As shown in Figure 18, there was some variability across MC+ MCOs in the statewide rate 
per 1,000 members of Pharmacy encounter claims compared to the rate for all MC+ MCOs 
(6,698.46 Pharmacy encounter claims per 1,000 members). Two MC+ MCOs (HealthCare USA, 
9139.85, z= 1.38, 95% Cl: 7869.70, 10409.99; p > .95 and Missouri Care, 8310.82, z = .90, 95% Cl: 
7040, 9580.97; p > .95) had significantly higher rates of Pharmacy encounter claims, while one MC+ 
MCO (Community Care Plus, 3783.14, z = -1.74; 95% Cl: 2512.99, 5053.29; p < .05) had a 


significantly lower rate of Pharmacy encounter claims per |,000 members than the rate for all MC+ 
MCOs. 


Figure 18. Pharmacy Encounter Claim Types per |,000 Members, January I, 2004 — March 31, 2004. 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2004 — March 31, 2004 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2004 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, enrollment for all Waivers, 
week ending March 26, 2004. 
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Figure 19 shows the proportion of claim types for each MC+ MCO based on the SMA encounter 
claims extract file. Community Care Plus had the highest proportion of Dental and Medical claim 
types relative to all other claim types; Mercy Health Plan had the highest proportion of the Inpatient 
claim types; and Blue Advantage Plus had the highest proportion of Home Health claim types. There 
were no patterns observed across MC+ Regions, suggesting that the variations are not related to 


member or provider practice characteristics. 


Figure 19. Proportion of Claim Types per MC+ MCO, January I, 2004 —March 31, 2004. 
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Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November! 1, 2004. 


To What Extent Do the Claims in the State Encounter Claims Database Reflect 
the Information Documented in the Medical Record? What is the Fault/Match 
Rate between State Encounter Claims and Medical Records? 


Table 40 shows the population (number of encounters), minimum required sample size, the number 
of encounters sampled, and the number and rate of records submitted for review. Of the 999,137 
Medical encounter claim types in the SMA encounter claims extract file for January |, 2004 through 
March 31, 2004, a total of 700 encounters (100 per MC+ MCO) were randomly selected. This was 
an oversample, as the minimum required sample size was 88 per MC+ MCO. Providers were 
requested to submit both medical records and claim forms/claim histories for review. For the 700 


selected encounters, there were 526 medical records (75.1%) and I61 claim forms (23.0%) 
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submitted for review. MC+ MCO submission rates ranged from 64.0% (Mercy Health Plan) to 
85.0% (Community Care Plus). Encounters for which no documentation was submitted were unable 
to be validated. Table 41 and Figure 20 show the results of the match for procedures. Across all 
MC+ MCOs, 53.3% of the medical records or claim forms contained matching procedure codes or 
descriptors. MC+ MCO match rates ranged from 44.0% (HealthCare USA) to 63.0% (Blue 
Advantage Plus). Three MC+ MCOs (Community Care Plus, 59.0%; z = .82, 95% Cl: 49.4, 68.6; p > 
.95; Missouri Care, 59.0%, z = .98, 95% Cl: 49.4, 68.6; p > .95; and Blue Advantage Plus, 63.0%, z 
=1.39, 95% Cl: 53.5, 72.5; p > .95) had match rates significantly higher than the rate for all MC+ 
MCOs; while one MC+ MCO ( HealthCare USA, 44.0%, z = -1.33, 95% Cl: 34.3, 53.7, p < .05) had 
significantly lower rates. By comparison, the State of Oregon reported a 99.4% match with medical 
records for the 1995/1996 review year (no methodology was reported).23 The fault rate for all 
MC+ MCOs for the procedure was 46.7%, with MC+ MCO fault rates ranging from 37.0% to 56.0%. 
When considering only the documentation submitted for review, the match rate for all MC+ MCOs 
for procedures was /0.9%, while the match rate for diagnoses was 73.8%. Thus, improved 


submission of medical records for review may improve the rate of validation of encounter data. 


Table 40. Encounter Data Validation Samples and Medical Record Submission Rate. 


Number Number 
Minimum Number Medical Claim 
Number Sample Encounters SX =tece) ge f-} Forms Submission 
Ob ee) Encounters Size Sampled Received Received Rate 
Community Care Plus 95,566 100 85 39 85.0% 
Mercy Health Plan 83,489 100 64 2/7 64.0% 
Health Care USA 445,349 100 67.0% 


Missouri Care 93,315 100 81.0% 
Family Health 

Partners 107,089 100 71.0% 
FirstGuard 98,364 100 74.0% 
Blue Advantage Plus 75,965 100 84.0% 
All MC+ MCOs 700 75.1% 





extract file for the period January |, 2004 through March 31, 2004. The minimum sample size is based on the validation of 
medical records for two dependent variables, the procedure code and the diagnosis code. Number Encounters Sampled = 
Number of unique encounters randomly sampled by the EQRO for medical record review validation. Number Medical 
Records Received = Number medical records submitted by MC+ MCO providers; Number Claim Forms Received = 
Number claim forms submitted by MC+ MCO providers; Submission Rate = Proportion of medical records submitted of 
the number of encounters sampled. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November | |, 2004. 
BHC, Inc. 2004 External Quality Review Validation of Encounter Data. 


*3 Center for Health Programs Development and Management, University of Maryland, & Actuarial Research 
Corporation (2003). A Guide to |lmplementing a Health-Based Risk-Adjusted Payment System for Medicaid 
Managed Care Programs. Centers for Medicare and Medicaid Services Contract No: 500-98-0004. 
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Table 41. Procedure Validation Rate. 








Number Rate Validated 
Number Medical roy am (=e | ere] | 
Encounters Records Records Actual Error (Fault) 

MC+ MCO Sampled Xela) AVL ace | Number Validated Received WE Vite ialelaM .t-ha= Rate ‘Ke UCL 
Community Care Plus 100 85 59 69.4% 59.0% 41.0% 49.36% 68.6% 
Mercy Health Plan 100 64 49 76.6% 49.0% 51.0% 39.20% 58.8% 
Health Care USA 100 67 44 65.7% 44.0% 56.0% 34.27% 53.7% 
Missouri Care 100 81 59 72.8% 59.0% 41.0% 49.36% 68.6% 
Family Health Partners 100 71 50 70.4% 50.0% 50.0% 40.20% 59.8% 
FirstGuard 100 74 49 66.2% 49.0% 51.0% 39.20% 58.8% 
Blue Advantage Plus 100 84 63 75.0% 63.0% 37.0% 53.54% 72.5% 
All MC+ MCOs 700 526 373 70.9% 53.3% 46.7% 49.6% 57.0% 


Note: Number Encounters Sampled = Number of unique encounters randomly sampled by the EQRO for medical record review validation; Number Medical Records Received 
= Number medical records submitted by MC+ MCO providers for validation; Number Validated = Number of encounters for which there was a similar or matching procedure 


code or description on the claim form, or adequate documentation in the medical record to support the procedure code as judged by a professional medical coder. Rate 
Validated of Medical Records Received = Number Validated/Number Medical Records Received; Actual Rate Validated = Number Validated/Number Encounters Sampled; LCL = 
Lower Confidence Limit at the 95% confidence level; UCL = Upper Confidence Limit at the 95% confidence level. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November | 1, 2004. BHC, Inc. 2004 External Quality Review Validation of 


Encounter Data. 
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Figure 20. Encounter Data Procedure Validation Rate, January |, 2004 —- March 31, 2004. 
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® Actual Validation Rate 59.0% 49.0% 44.0% 59.0% 50.0% 49.0% 63.0% 53.3% 
MC+ MCO 





Note: * Indicates values are significant at the 95% level of significance, two-tailed z-test. See corresponding tables for 95% 
confidence intervals. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November | |, 2004. 
BHC, Inc. 2004 External Quality Review Validation of Encounter Data. 


For the validation of the diagnosis, 388 (55.4%) matched the diagnosis found in the SMA encounter 
claims extract file across all MC+ MCOs (see Table 42 and Figure 21). MC+ MCO match rates 
ranged from 45.0% (Family Health Partners) to 68.0% (Community Care Plus) of the medical 
records or claim forms for diagnosis codes or descriptors. Two MC+ MCOs (Community Care 
Plus, 59.0%, z =1.58, 95% Cl: 58.9, 77.1; p > .95; and Missouri Care, 61%, z =1.58, 95% Cl: 51.4, 70.6; 
p > .95) had match rates significantly higher than the rate for all MC+ MCOs; while three MC+ 
MCOs (Mercy Health Plan, 49.0%, z = - .81, Cl: 39.2, 58.8; p < .05; Health Care USA, 50%, z = - .68, 
Cl: 40.2, 59.8; p < .05; and Family Health Partners, 45.0%; z = -1.31, Cl: 35.2, 54.8, p < .05) had 
significantly lower rates. By comparison, the State of Oregon reported an accuracy rate of 88.2% 
for the 5-digit diagnosis code, and 91.0% for the 3-digit diagnosis code for the 1995/1996 review 
year (no methodology was reported).24 The fault rate for all MC+ MCOs on the diagnosis was 


44.6%, with MC+ MCO fault rates ranging from 32.0% to 55.0%. 


*4 Center for Health Programs Development and Management, University of Maryland, & Actuarial Research 
Corporation (2003). A Guide to |lmplementing a Health-Based Risk-Adjusted Payment System for Medicaid 
Managed Care Programs. Centers for Medicare and Medicaid Services Contract No: 500-98-0004. 
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Table 42. Diagnosis Validation Rate. 


Number 


Encounters 


MC+ MCO 
Community Care Plus 
Mercy Health Plan 
Health Care USA 
Missouri Care 
Family Health Partners 
FirstGuard 
Blue Advantage Plus 
All MC+ MCOs 





Requested 


100 
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100 
700 


Number 
| (=xe | (er-] | 

| X=Yele) ae [> 
Received 

85 

64 

67 

81 

71 

74 

84 

526 


68 
49 
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61 
45 
56 
59 
388 


Rate Validated 
ro} mm C=Xe | Core] | 
X={elo) we [- 
Received 


80.0% 
76.6% 
74.6% 
715.3% 
63.4% 
15.7% 
70.2% 
73.8% 


Actual 
WE-Vitef-hatelalm .t-1 a= 


68.0% 
49.0% 
50.0% 
61.0% 
45.0% 
56.0% 
59.0% 
55.4% 


Saaelamear-leiie) 
Rate 


32.0% 
51.0% 
50.0% 
39.0% 
55.0% 
44.0% 
41.0% 
44.6% 


LCL 

58.9% 
39.2% 
40.2% 
51.4% 
35.2% 
46.3% 
49.4% 
51.7% 


Welk 


771.1% 
58.8% 
59.8% 
70.6% 
54.8% 
65.7% 
68.6% 
59.1% 


Note: Number Encounters Sampled = Number of unique encounters randomly sampled by the EQRO for medical record review validation; Number Medical Records Received 
= Number medical records submitted by MC+ MCO providers for validation; Number Validated = Number of encounters for which there was a matching diagnosis code, 
documentation or description in the medical record or on the claim form. Rate Validated of Medical Records Received = Number Validated/Number Medical Records Received; 
Actual Rate Validated = Number Validated/Number Encounters Sampled; LCL = Lower Confidence Limit at the 95% confidence level; UCL = Upper Confidence Limit at the 95% 


confidence level. 


Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November | 1, 2004. BHC, Inc. 2004 External Quality Review Validation of 


Encounter Data. 


Performance Management Solutions Group 


a Division of Behavioral Health Concepts, Inc. 


T: 573.446.0405 
F: 573.446.1816 
E-mail: info@bhcinfo.com 


2/16 Forum Blvd., Suite 4 
Columbia, MO 65203 
www. bhcinfo.com/PMSG 


133 


EXTERNAL QUALITY REVIEW OF MissouR!I MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Figure 21. Encounter Data Diagnosis Validation Rate, January |, 2004 — March 31, 2004. 
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Note: * Indicates values are significant at the 95% level of significance, two-tailed z-test. See corresponding tables for 95% 
confidence intervals. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, November | 1, 2004. BHC, 
Inc. 2004 External Quality Review Validation of Encounter Data. 


What Types of Errors Were Noted? 

An error analysis for procedure and diagnosis codes and descriptors was conducted. For the diagnosis 
code in the medical record, the reasons for diagnosis codes not matching the SMA encounter claims 
extract file were missing (n = 630) or incorrect information (n = 13). For the diagnosis description in 
the medical record, the reasons for diagnoses not matching the SMA encounter claims extract file were 
missing or illegible information (n = 333), or no match with the description of the symptoms based on 
the information in the medical record (n = 59). For the diagnosis code on the claim form, the reasons 
for diagnosis codes not matching the SMA encounter claims extract file were missing or illegible 
information (n = 563), or possible data entry or recording error by the provider (n = 7). The reasons 
for the diagnosis descriptor on the claim form not matching the SMA encounter claims extract file 


included missing or illegible information (n = 4/76) and incorrect codes (n = 4). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA encounter 
claims extract file not being supported by documentation in the medical record were missing 


information (n = 59), downcoding (billing a service that is reimbursed at a lower rate or for less time 
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than actually spent with the patient; n = 24), incorrect codes (n = 49), upcoding (billing a service that is 
reimbursed at a higher rate or for more time than actually spent with the patient; n = 38), illegible 
information (n = 6), and not enough information for coding (n = |7). For the procedure code on the 
claim form, the reasons for procedure codes not matching the SMA encounter claims extract file were 
missing or illegible information (n = 559) or incorrect codes (n = I 1). For the procedure description 
on the claim form, the reasons for procedure codes not matching the SMA encounter claims extract 


file were missing or illegible information (n = 619), and incorrect descriptors (n = 6). 


The analysis of the third objective, to compare the SMA encounter claims extract file to MC+ MCO 
paid and unpaid claims, was unable to be accomplished due to the variability and quality of files 
submitted by MC+ MCOs. After review by the EQRO Project Director of the files submitted by the 
MC+ MCOs for the encounter data validation, a summary report was provided to the SMA on 
February 8, 2005. These findings were used to address the questions posed about data quality and 
submission of files by MC+ MCOs. 


What Problems Are There With How Files Are Compiled and Submitted by the 
MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claims extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to the 
SMA encounter claims extract file. MC+ MCOs were requested to submit data in national standard file 
layouts for the MC+ Managed Care Members represented in the encounter claim sample selected by 
the EQRO for validation. All MC+ MCOs data submitted files in response to the EQRO request for 
ASCII format with acceptable field delimiters, in the file layout required by the SMA prior to October 
2003. The EQRO received feedback from MC+ MCOs that they were unable to submit files in the file 
layout prior to October 2003 and were unable to submit unpaid claims. MC+ MCOs were then 
instructed to submit files in the three national standard file layouts (e.g., NSF/CMS 1500, UB-92) in 
either the pre- or post- October 2003 file layout. MC+ MCOs indicated they were unable to submit 
unpaid claims in the national standard formats and were instructed by the EQRO to submit the files in 


available file layouts, in ASCII format with acceptable delimiters and adequate documentation for the 


files. 
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Numerous files were submitted by MC+ MCOs in response to this request. Little to no 
documentation was provided for non-standard file layouts and the contents of each file. Over the 
course of several months, the EQRO endeavored to identify the file layouts and contents of each of the 
MC+ MCO files and assess the viability of a valid analysis comparing the files to the SMA encounter 
claims extract file. Although several MC+ MCOs were able to submit some files that were able to be 
loaded for analysis, there was no way to know whether all types of encounter claims (Medical, Dental, 
Inpatient, Outpatient Hospital, Home Health, and Pharmacy) were included in the files submitted. The 


EQRO determined that any analysis based on the data provided would not likely be valid. 


What Are the Data Quality Issues Associated With the Processing of Encounter 
Data? 


There were several data quality issues with SMA and MC+ MCO encounter data identified during the 
course of conducting the EQRO. These issues are primarily related to the manner in which data are 
extracted from single and multiple databases. For the SMA encounter claims extract file, it was not 
possible to validate procedure code modifiers due to misalignment of data fields. MC+ MCOs were 
unable to submit unpaid encounter claims from the time period of interest, as they do not retain this 
data and cannot program unpaid encounter claims into national standard file layouts. Others identified 
limited data processing and programming resources for the encounter claim data request. This makes 
it untenable to identify errors of commission (unpaid claims in the SMA encounter claims file). Some 


examples of the findings from review of MC+ MCO encounter data file submissions are provided 


below. 

e Files which were judged to be unpaid encounters based on the file were unable to be loaded using any of the 
national standard file layouts. 

e One MC+ MCO submitted all three file layouts in one file. Although the three layouts were able to be 
identified, not all records were able to be loaded for analysis. This occurred primarily with the UB-92 file 
layouts. 

e Records in some files appeared truncated. 

e Other files were submitted in Structured Query Language or spreadsheet formats. 
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CONCLUSIONS 


Strengths 


|. All Dental, Home Health and Pharmacy claim type fields examined were |00.00% complete, 
accurate and valid for all MC+ MCOs. The SMA encounter claims data critical fields examined for 
accepted and paid claims of this type are valid for analysis. The next step would be to compare 
MC+ MCO paid claims to the SMA encounter claims database to identify the level of completeness 
of this data. 


2. Missouri Care had 100.00% complete, accurate and valid data for all critical fields examined in the 
Outpatient Hospital claim type. 


3. For the Medical claim type, all critical fields examined were |00.00% complete and accurate for all 
MC+ MCOs. 


4. For all MC+ MCOs, the Inpatient claim type critical fields examined were |00.00% complete, with 
the correct length of data 100.00% of the time. The Claim Type, Recipient ID, Admission Type, Bill 
Type, and Diagnosis Code fields were 100.00% valid for all MC+ MCOs. 


5. For all MC+ MCOs, the Outpatient Hospital claim type critical fields examined were 100.00% 
complete. 


6. For all MC+ MCOs, the Outpatient Hospital Claim Type field, Recipient ID, First Date of Service, 
Last Date of Service, Units of Service, and the first Diagnosis Code fields were 100.00% complete, 
accurate and valid. 


7. The examination of the level, volume, and consistency of services found significant variability 
between MC+ MCOs in the rate of each type of claim (Medical, Dental, Inpatient, Outpatient 
Hospital, Home Health, and Pharmacy), with no patterns of variation noted by MC+ Managed Care 
Region or type of MC+ MCO. 


8. Of the medical records received for review, the rates of matching on the procedure and diagnosis 
were /3.8% and /0.9%, respectively. However, the medical records not submitted were unable to 
be validated. 


Areas for Improvement 


|. For the Medical claim type, there were invalid values for the Outpatient Units of Service and 
Outpatient Procedure Code fields. Mercy Health Plan reported invalid values of “Q0000” for 
Outpatient Units of Service; and there were invalid procedure codes from Community Care Plus, 
Mercy Health Plan, Missouri Care, HealthCare USA, Family Health Partners, and FirstGuard. 


2. For the Inpatient claim type critical fields, the Admission Date, Discharge Date, Patient Status, and 
Revenue Code fields contained invalid values. Invalid Admission Dates from | 2/06/2003 - 
|2/31/2003 were present for Community Care Plus; from I 1/11/2003 - 12/31/2003 for Missouri 
Care; | 1/16/2003 — 12/31/2003 for Family Health Partners; 01/22/2003 — 12/31/2003 for 
HealthCare USA; | 1/30/2003 — 12/31/2003 for Mercy Health Plan; and 09/13/2003 for Blue 
Advantage Plus. These findings may be a function of the manner in which the SMA encounter 
claims extract file was constructed. Invalid Discharge Date fields consisted of entries of 
“99999999” from Community Care Plus, Mercy Health Plan, HealthCare USA, Missouri Care, 
FirstGuard, and Blue Advantage Plus. The Patient Status field consisted of invalid codes of “00”, 
“63”, and “16” for HealthCare USA, Missouri Care, and FirstGuard. Invalid Revenue Code fields 
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included blank fields and an invalid value of “080” (Mercy Health Plan, HealthCare USA, Missouri 
Care, Family Health Partners, and FirstGuard). The Units of Service field contained blank fields and 
invalid values of “QOQ000” (Community Care Plus, Mercy Health Plan, HealthCare USA, Missouri 
Care, and Family Health Partners). 


3. For the Outpatient Hospital claim type, critical fields with invalid data were the Outpatient 
Procedure and Revenue Code fields. Invalid Outpatient Procedure Code fields included invalid 
values of “0000”, values less than five characters, and invalid procedure codes (Community Care 
Plus, Family Health Partners, and FirstGuard). The Revenue Code field contained invalid values of 
“00000”, “0” —"035” (Community Care Plus, HealthCare USA, and Family Health Partners). 


4. The match rates between the SMA database and MC+ MCO medical records for Medical claim type 
procedures and diagnoses were 53.3% and 55.4%, respectively. There was significant difficulty 
identifying the rendering providers, as the SMA database does not accept this information, and 
MC+ MCOs had difficulty providing reliable contact information for medical record requests. The 
ability to link the rendering provider with the encounter being sampled is critical for any validation 
process requiring medical record review. Medical records that did not have diagnosis or procedure 
codes that matched the SMA encounter claims extract file were in error primarily due to missing 
or illegible information. 


5. Clarification on the potential source of variability in each claim type across MC+ MCOs is needed 
to rule out the possibility of incomplete data or identify data issues to target for improvement. 


6. Comparison data from other State Medicaid programs using similar claim types and with similar 
benefit packages is needed. 


7. Due to the variability in the data submitted by MC+ MCOs, the present validation process was not 
able to address the level of completeness of the SMA encounter claims database to identify material 
omissions in the SMA encounter claims database for claims paid by the MC+ MCOs. 


8. The completeness, accuracy, and validity of the procedure modifier fields was unable to be validated 
due to EQRO concerns about the representativeness of the SMA encounter claims extract file. 
MC+ MCOs were not uniformly able to submit paid and unpaid claims; or revert to file layouts to 
those in effect prior to the changes made in October 2003. 


9. The accuracy and completeness of the SMA encounter claims database was unable to be evaluated 
based on the MC+ MCO encounter claim file submissions. MC+ MCOs are unable to identify 
unpaid claims. 
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Recommendations 


|. The SMA should prioritize examination of the level of completeness of the SMA Dental, Home 
Health, and Pharmacy paid encounter claim types relative the claims submitted by MC+ MCOs. 
These claim types are likely to be more complete and valid sources of data on which to base initial 
rate setting. This can be done once an identified method of obtaining paid and unpaid claims form 
MC+ MCOs is developed. 


2. It is recommended that the SMA institute additional edits for the Medical, Inpatient and Outpatient 
Hospital claim types to edit claims with blank fields or dummy values (e.g., “O00” and “99999999”’). 
Edits for discontinued procedure codes and local codes should also be implemented. 


3. It is recommended that MC+ MCOs examine potential data quality issues associated with the 
variability in identification of eligible members and administrative hits for the performance measure 
validation process. 


4. MC+ MCOs are required to submit Revenue Codes on the Outpatient Hospital (UB-92) claim file 
layout regardless of the Procedure Code. The SMA should institute edits for missing data. 


5. It is recommended that future encounter data validation efforts explore the variability in the 
number of members who are enrolled with capitated providers to assess whether or not the 
variability in the proportion of each claim type or rate of encounters is related to the payment 
method. 


6. The SMA should continue to provide timely feedback to MC+ MCOs regarding the rate of 
acceptance of each claim type and the types of errors associated with rejected claims. 


7. To improve the completeness of encounter data submission by providers, it is recommended that 
MC+ MCOs discontinue capitated payments in favor of discounted Fee-for-Service arrangements as 
contracts with provider groups are renewed. MC+ MCOs should emphasize to providers the 
importance of documenting diagnosis and procedure information in the medical record. 


8. Encounter data validation efforts should examine the rate of claims by date of service, date of 
payment, and claim type on a monthly basis, for at least one year. This would identify the level of 
consistency of accurate claims submission, permit the identification of patterns and issues 
associated with any changes in system level edits or MC+ MCO information system changes, and 
identify claims submission timeliness of MC+ MCOs. 


9. Additional analysis on the rate of consistency of services should examine demographic (e.g., age and 
gender distribution), epidemiological (diagnostic variables), and service delivery (e.g., number of 
users per month, rate of procedures or claim types, units of service rates) characteristics to explain 
variation across MC+ MCOs or Regions. 


10. Medical record reviews should continue to be targeted toward validation of diagnosis and 
procedure codes. 


I1. The SMA should provide to the EQRO individual claim identifiers (ICNs) when extracting 
encounter claims for medical record review purposes. 


12. When targeting fields for validation, the SMA should provide the EQRO with the parameters to be 
used for analyzing the completeness, accuracy, and validity of the data fields. This was conducted 
by the EQRO this year, but ideally should be developed by the SMA to ensure credible analysis and 
conformity to SMA requirements for MC+ MCOs. This should be supplied to and discussed with 
the EQRO prior to the initiation of validation activities. 
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13. The SMA should clarify the expectations for MC+ MCQs in the level of completeness, accuracy, 
and validity and which data fields are required (e.g., Diagnosis Code fields 2 through 5); provide 
timely feedback to MC+ MCOs when standards are not met; and develop corrective action plans 
when standards are not met within a reasonable amount of time established by the SMA. 


14. The SMA should provide more time for MC+ MCOs to implement data file layout conversions. 
This transition may have accounted for the poor quality of data and availability of MC+ MCO staff 
to comply with EQRO data submissions. 


I5. MC+ MCOs will need to submit data to the EQRO in requested formats, using the field names and 
file formats requested. Given the resources involved in programming extract files for encounter 
validation, MC+ MCOs will need at least one month to fulfill requests. For the encounter data 
validation, it is recommended that paid claims be submitted in the current file layouts as required 
by the SMA and indicated in the Health Plan Record Layout Manual. These should all be submitted 
in one format, such as comma-delimited text file formats. Other formats and incomplete data 
should be treated as missing or invalid. It is recommended that the SMA and MC+ MCOs identify 
valid and consistent file layouts and definitions for paid and unpaid encounter claims extract files to 
be validated. 


16. Another issue associated with file submissions was related to the submission of provider addresses, 
the layouts, and the accuracy of addresses for the rendering provider. MC+ MCOs should use 
standard database formats for provider network databases and addresses (e.g., US Postal service 
standards?>). MC+ MCOs should submit medical records for encounter data and performance 
measure validation directly to the EQRO due to the error associated with identifying providers and 
provider addresses. The elimination of the process of requesting addresses from MC+ MCOs for 
the sampled encounters would allow more time for medical record collection and submission. 


17. The American Health Information Management Association (AHIMA) has published, “A Checklist 
to Assess Data Quality Management Efforts”2¢ that should be reviewed by MC+ MCOs. Data 
management and data quality efforts should include the ability to provide the SMA and the EQRO 
with valid encounter claims data for quality improvement and encounter data corrective action 
planning purposes. 


18. It is recommended that the SMA identify similar State Medicaid managed care programs (e.g., 
recipient, eligibility, and benefit characteristics), obtain findings of encounter data validation 
activities for external comparison, and use the present findings as a baseline for assessing 
improvement in the validity of encounter data for the MC+ Managed Care Program over time. 


2° US Post Office (2000). Postal Addressing Standards. Publication 28. 
http: //pe.usps.gov/text/pub28/welcome.htm, retrieved 05/26/2005. 
2° American Health Information Management Association (1998). Practice Brief: A Checklist to Assess Data 


Quality Management Efforts. Journal of AHIMA. http://library.ahima.org/xpedio/groups/documents/ahima/pub..., 
retrieved 05/09/2005. 
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SECTION FOUR: MC+ MCO COMPLIANCE WITH MANAGED 


CARE REGULATIONS 





Purpose and Objectives 


The External Quality Review (EQR) is conducted annually in accordance with the “Medicaid Program; 
External Quality Review of the Medicaid Managed Care Organizations: Final Rule, 42 CFR Part 438, 
Subpart E.”” The objective of this portion of the review was to analyze and evaluate the MC+ Managed 
Care Organizations (MC+ MCOs) to assess their level of compliance with federal regulations regarding 
the quality, timeliness and access to health care services. To complete this, the “Monitoring Medicaid 
Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A Protocol for 
Determining Compliance with Medicaid Managed Care Regulations” (Compliance Protocol) 
requirements were applied to the review process. The specific objectives of the compliance review 
were to assess MC+ MCO compliance with federal Medicaid managed care regulations; MC+ MCO 
compliance with the State Quality Strategy; the quality, timeliness and access to services; and the 


progress of MC+ MCOs from the previous year. 


Technical Methods 


PLANNING COMPLIANCE MONITORING ACTIVITIES 

Establishing Contact with the MC+ MCOs 

All MC+ MCOs were contacted during October 2004. Conversations were held with QI/UM 
Coordinators, the Medicaid Plan Administrator, or their designee about the requirements for the EQR. 
Each MC+ MCO was sent information on all the protocols. A conference call was scheduled for the 
end of October 2004. During this conference call, all aspects of the EQR were discussed and details 
provided regarding all data submissions that would be required for the EQR. Follow-up telephone 
contacts and written communications occurred providing information and answering questions before 


and after the on-site review. 


Gathering Information on the MC+ MCO Characteristics 
During 2004, there were seven MC+ MCOs contracted with the State Medicaid Agency (SMA; Missouri 
Department of Social Services, Division of Medical Services; DMS) to provide MC+ Medicaid managed 


care in three Regions of Missouri. The Eastern MC+ Region, which included St. Louis City, St. Louis 
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County, and eight surrounding counties had access to three MC+ MCOs: Community Care Plus 
(CCP), HealthCare USA (HCUSA), and Mercy Health Plan (MHP). The Western MC+ Region, which 
included the Kansas City Area/Jackson County and eight surrounding counties, was served by four MC+ 
MCOs: Family Health Partners (FHP), FirstGuard, Blue Advantage Plus (BA+), and HealthCare USA 
(HCUSA). The Central MC+ Region included eighteen counties in the center of the state. These 
counties were served by two MC+ MCOs: Missouri Care (MOCare), and HealthCare USA. 
HealthCare USA operated in all three MC+ Regions. 


Determining the Length of Visit and Dates 
On-site reviews were conducted in one day, with several reviewers conducting interviews and activities 
concurrently. Interviews, presentations and document reviews were scheduled throughout the day 
utilizing different team members for Validating Performance Measures, Validating Performance 
Improvement Projects, and Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid 
Inpatient Health Plans (PIHPs). The time frame for on-site reviews was approved by the SMA before 
scheduling each MC= MCO. The first week was spent reviewing the MC+ MCOs in the Eastern MC+ 
Region. The second review week was spent in the Western MC+ Region. A final visit occurred with 
the MC+ MCO in the Central MC+ Region. Site visits occurred during the month of March 2005 on 
the following schedule: 

e March |, 2005 — Community Care Plus 

e March 2, 2005 — Mercy Health Plan 

e March 3, 2005 — HealthCare USA 

e March 14, 2005 — Family Health Partners 

e March 15, 2005 — Blue Advantage Plus 

e March 16, 2005 — FirstGuard 

e March 23, 2005 — Missouri Care 


Reviewers 

Three reviewers conducted the Compliance Protocol activities, including interviews and document 
review. The External Quality Review Organization (EQRO) Project Director conducted four prior 
EQRs for the SMA and several evaluations of the MC+ For Kids 1115 Waiver Program. She had in- 
depth knowledge about the MC+ Managed Care Program and MC+ MCO operations. The second 
reviewer participated in three previous MC+ Managed Care Program EQRs and on-site visits. This 
reviewer was knowledgeable about the MC+ Managed Care Program through her experience as a 


former SMA employee responsible for quality assessment and improvement, an RN, and consultant. 
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The third reviewer had experience with the MC+ Managed Care Program implementation and 
operations, interviewing, program analysis, and Medicaid managed care programs in several other 
states. All reviewers were familiar with the federal regulations and the manner in which these were 


operationalized by the MC+ Managed Care Program prior to refining the protocols. 


Establishing an Agenda for the Visit 

An agenda was developed to maximize the use of available time, while ensuring that all relevant 
individuals necessary were interviewed. A sample schedule was developed that specified times for the 
entrance conference, document review, Validating Performance Improvement Project evaluation, 
Validating Performance Measures review, the interview schedule for the Compliance Protocol, and the 
exit conference. A coordinated effort with each MC+ MCO, grouping individuals together for 
interviews based on functions performed, was developed. The schedule for the on-site reviews was 
approved by the SMA in advance. Arrangements were then made directly with each MC+ MCO. 


Appendix 7 provides a sample agenda for on-site reviews. 


Providing Preparation Instructions and Guidance to the MC+ MCOs 

A letter (see Appendix 7) was sent to each MC+ MCO indicating the information and documents 
required on-site and the individuals who were requested to attend each interview session. The MC+ 
MCOs ensured that staff was available and that all requested documentation was present at the 


beginning of the on-site review day. 


OBTAINING BACKGROUND INFORMATION FROM THE STATE MEDICAID AGENCY 
Interviews and meetings occurred with individuals from the SMA from September 2004 through March 
2005 to prepare for the on-site review and obtain information relevant to the review prior to the on- 
site visits. Individuals from the SMA included in these meetings were: 


e Sandra Levels, Director — Program Management 

e ~—_ Billie Waite, Legal Counsel 

e Judith Muck, Assistant Deputy Director 

e Susan Bishop, Auditor III 

e Andrea Smith, Quality Program Liaison 

e Kimberly Carter, MC+ Managed Care QA & | Manager 


Following EQRO review of SMA Compliance findings for each MC+ MCO, EQRO personnel met with 
the SMA MC+ Managed Care QA & | Manager on February 8 and 9, 2005. Clarification of the MC+ 
Medicaid Managed Care contract compliance information shared by the SMA was obtained, and 


additional MC+ MCO documentation was reviewed to ensure that reviews measured the standard of 
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care expected by the SMA in each MC+ Managed Care Region. The SMA staff were in the process of 
reviewing prior authorization policies for each MC+ MCO and a determination of contract compliance 
was not yet finalized by the SMA. Therefore, the EQRO reviewed prior authorization practices during 
the on-site visits to accurately assess compliance with federal regulatory provisions. SMA 
expectations, requirements, and decisions specific to the MC+ Managed Care Program were identified 


during these meetings. 


DOCUMENT REVIEW 

Documents chosen for review were those that best demonstrate the MC+ MCOs’ ability to meet 
federal regulations. Certain documents, such as the Member Handbook, provided evidence of 
communication to members about a broad spectrum of information from enrollee rights through the 
grievance and appeal process. Provider handbooks and provider agreements were also reviewed to 
ensure that consistent information was shared regarding enrollee rights and responsibilities. SMA MC+ 
Medicaid Managed Care contract compliance worksheets, specific policies yet to be approved by the 
SMA, and other information requested from or shared by the MC+ MCOs were reviewed to verify the 
presence or absence of evidence that regulations were met. When it was found that specific 
regulations were not met or Partially Met , additional documents were requested by each MC+ MCO 
and interview questions were developed to address the areas for which compliance was not able to be 


established through the pre-site document review process. 


The following documents were reviewed for all MC+ MCOs: 


e State contract compliance ratings from 2004 and updated policies accepted through February 2005 
e = Results, findings, and follow-up from 2003 External Quality Review 
e 2003 Annual MC+ MCO Evaluation, submitted April 2004 


CONDUCTING INTERVIEWS 

After completing the document review, it was necessary to determine how policies were implemented 
by the MC+ MCOs. Interviews on-site with the staff enabled reviewers to obtain a clearer picture of 
the degree of compliance achieved by each MC+ MCO and often revealed the extent to which the 
MC+ MCO was implementing policies. Additionally, interviews provided reviewers with the 
opportunity to explore issues not fully addressed in the documentation, including follow-up areas from 
the previous EQRO evaluation. A site visit questionnaire was developed for each MC+ MCO based on 


the MC+ MCO’s compliance with the MC+ Medicaid Managed Care contract, issues identified for 
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clarification, information presented in the 2004 Annual Report, document review, and follow-up items 


from the previous External Quality Review. 


COLLECTING ACCESSORY INFORMATION 

Additional information used in completing the compliance determinations included discussions with the 
EQR reviewers and MC+ MCO QI/UM staff regarding management information systems, Validating 
Encounter Data, Validating Performance Measures, and Validating Performance Improvement Projects. 
The review evaluated information from these sources to validate MC+ MCO compliance with the 
pertinent regulatory provisions in the Compliance Protocol. These findings were documented on the 


BHC MC+ MCO Compliance Review Scoring Form and used to make final rating recommendations. 


ANALYZING AND COMPILING FINDINGS 

The review process included gathering information and documentation from the SMA about policy 
submission and the MC+ Medicaid Managed Care contract compliance for each MC+ MCO, followed 
by a detailed review of this information and how it related to compliance with the federal regulations. 
The next step included preparing MC+ MCO specific questions based on the need to investigate if 
practice existed in areas where approved policy was not available, and if local policy and procedures 
were in use if official approved policy was not complete. The interview responses obtained during the 
on-site review were then analyzed in combination with the evaluation of all documentation obtained 
and reviewed on-site. This information was assessed and re-reviewed to be translated into 
recommended compliance ratings for each regulatory provision, and were recorded on the MC+ MCO 


Scoring Form. Comments were recorded to identify positive practices and deficiencies. 


REPORTING TO THE STATE MEDICAID AGENCY 

Individual worksheets were submitted to the SMA in late March and early April 2005 for review and an 
evaluation of the recommended rating determinations. Some discussion occurred with staff from the 
SMA to clarify information and the rationale for decisions. The SMA approved the process and ratings 
on April 19, 2005, but wanted to ensure that the worksheets would include sufficient detail to 
substantiate the ratings. Additional detail for each MC+ MCO and item not meeting compliance was 
provided, and final worksheets were submitted to the SMA. The final ratings are presented in this 


report. 


COMPLIANCE RATINGS 
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In January 2005, a worksheet identifying each federal regulation to be reviewed by the EQRO along 
with a crosswalk referencing applicable SMA MC+ Medicaid Managed Care contract compliance 
references was submitted to the SMA for review and comment. Given that there was no rating system 
developed by the SMA, the EQRO recommended a three-point scale (“Met”, “Partially Met’, “Not 
Met’’) for measuring compliance and provided preliminary ratings to be reviewed and finalized by SMA. 
This document, with suggestions for MC+ Medicaid Managed Care contract references to be utilized in 
making the recommendations for final ratings for the MC+ MCOs was returned to the EQRO on 


March 21, 2005. 


Appendix 7 contains the BHC MCO Compliance Review Scoring Form worksheet that was used for all 
MC+ MCOs. In some instances, additional MC+ Medicaid Managed Care contract compliance sections 
were included in determining the final rating. Compliance ratings considered SMA contract compliance 
review findings, MC+ MCO policy, ancillary documentation, and MC+ MCO practices in making a final 
recommendation for ratings. In many instances, the SMA MC+ Medicaid Managed Care contract 
compliance tool rated a contract section as “Met” if policies were submitted, even if the policy had not 
been reviewed and finally approved. Ratings were based upon a three-point scale approved by the 


SMA. This scale allowed for credit when a requirement was Partially Met . Ratings were defined as: 


All documentation listed under a regulatory provision, or one of its 
components was present. MC+ MCO staff were able to provide responses to 
reviewers that were consistent with one another and the available 
documentation. Evidence was found and could be established that the MC+ 
MCO was in full compliance with regulatory provisions. 


Partially Met : There was evidence of compliance with all documentation requirements, but 
staff were unable to consistently articulate processes during interviews; or 
documentation was incomplete or inconsistent with practice. 


Incomplete documentation was present and staff had little to no knowledge 
of processes or issues addressed by the regulatory provision. 





Findings 


ENROLLEE RIGHTS AND PROTECTIONS 

Subpart C of the regulatory provisions for Medicaid managed care (Enrollee Rights and Protections) 
sets forth |3 requirements of MCOs for the provision of information to enrollees in an understandable 
form and language; written policies regarding enrollee rights and assurance that staff and contractors 
take them into account when providing services; and the provision of access to care in compliance with 


requirements for payment and no liability of payment for enrollees. There were very few items across 
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MC+ MCOs (3 items) that were Not Met for Enrollee Rights and Protections (see Table 43). Across 
MC+ MCOs, 54.9% of the regulations were Met. MC+ MCOs Met 15.4% (Community Care Plus, 


Mercy Health Plan) to 100% (First Guard) of the requirements for Enrollee Rights and Protections. 
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: Mc+MCO : | 

P = HCUSA secral er | co lose 
1 1 1 1 2 2 
2 2 2 2 


Table 43. Subpart C: Enrollee Rights and Protections. 








; | 


All MCOs 





Rate 

Federal Regulation C Not Met Met 
438.100(a) Enrollee Rights: General Rule 1 0 28.6% 
438.10(b) Enrollee Rights: Information Requirements 0 1 1 4 1 57.1% 
438.10(c)(3) Alternative Language: Prevalent Language 1 2 2 2 2 2 2 6 0 85.7% 
438.10(c)(4,5) Language and Format: Interpreter 
Services 1 1 1 1 2 2 2 3 0 42.9% 
438.10(d)(1)(i) Information Requirements: Format/Easily 
Understood 1 1 2 2 2 2 2 5 0 71.4% 
438.10(d)(1)(ii)and (2) Information Requirements: Format 
Visually Impaired, and Limited Reading Proficiency 0 1 1 2 2 2 2 4 1 57.1% 
438.10(f) Information for All Enrollees: Free Choice, etc. 1 1 1 2 2 2 1 3 0 42.9% 
438.10 (g) Information to Enrollees: Specifics/Physician 
Incentive Plans 2 1 2 2 2 2 2 6 0 85.7% 
438.10(i) Special Rules: Liability for Payment/Cost 
Sharing 1 1 1 1 2 2 1 2 0 28.6% 
438.100(b)(2) (ili) Enrollee Rights: Provider-Enrollee 
Communications 1 2 2 2 2 2 2 6 0 85.7% 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance 
Directives 1 1 1 1 1 2 2 2 0 28.6% 
438.100(b)(3) Right to Services 1 0 1 1 1 2 2 2 1 28.6% 
438.100(d) Compliance with Other Federal/State Laws 2 1 2 1 2 2 2 5 2 0 71.4% 
Number Met 2 2 5 7 10 13 ll 3 
Number Partially Met 9 10 8 6 3 0 2 
Number Not Met 2 1 0 0 0 0 0 
Rate Met 15.4% 15.4% 38.5% 53.8% 76.9% 100.0% 84.6% 


Note: 0 = Not Met; |= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient 
Health Plans (PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 
External Quality Review Monitoring MCOs Protocols. 
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The MC+ MCOs had procedures in place to work across departments to resolve problems that 
members brought to their attention. They assisted members in obtaining timely appointments, 
finding transportation, and in locating community resources. One MC+ MCO (HealthCare USA) 
had designated staff in the case management section that were identified as “Outreach 
Coordinators.” These staff members made a concerted effort to engage any member who had a 
special healthcare need. They ensured that these members were appropriately identified and 


received the services required. 


All MC+ MCOs had policies and practices that reflected the members’ right to a second opinion, 
and a third opinion if the first two disagreed [438.206(b)(3)]. A second area where all MC+ MCOs 
were fully compliant was in Provider Cultural Competency [438.206(c)(2)]. The MC+ MCOs made 
solid efforts to insure that providers understood or became aware of cultural issues that may affect 
healthcare decisions for members. They were respectful of language differences and made a 
concerted effort to offer choices of providers who spoke their language to members whose primary 


language was not English. 


Five of the seven MC+ MCOs (Community Care Plus, Mercy Health Plan, HealthCare USA, Family 
Health Partners, and FirstGuard) contracted for services with behavioral health organizations 
(BHOs). One MC+ MCO (Missouri Care) was in the process of developing internal behavioral 
health services. Behavioral health services evolved into an important resource for members with 
behavioral or mental health needs. A number of MC+ MCO BHOs (Community Care Plus, Mercy 
Health Care, HealthCare USA) approved in-home services to reach members who would not attend 
appointments in an office setting. This not only ensured that the member obtained the help they 
needed, but also prevented a provider from scheduling an appointment that would not be kept. 
Providers reported gaining valuable insight into environmental issues that contributed to the 
members’ behavioral or mental health problems through the use of in-home services. One MCt+ 
MCO BHO (Unity Management Mental Health) that contracts with Community Care Plus and 
Mercy Health Plan indicated that they contract with providers who specialized in serving the 
Medicaid population. One MC+ MCO BHO (New Directions Behavioral Health) contracted with a 


provider agency that delivered short-term intensive in-home services to avert crises that may lead 
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to inpatient treatment or other serious family problems such as out-of-home placement for the 
children. The MC+ MCOs and BHOs described a number of interventions that met members’ 
needs, but were extraordinary in normal Medicaid managed care systems. This reflected a level of 


performance indicative of their strong commitment to access to services for all members. 


Problems in the area of access to care were evident. Required documentation and approved 
policies did not exist in all areas. Some policies were not developed, some had not been through 
the MC+ MCO internal approval process, and some were not yet submitted to the SMA. Two of 
the seven (28.6%) MC+ MCOs Met regulations for having written policies in place to ensure 
consistent application of review criteria for the authorization of services [438.210(b)]; and for the 
prohibition against limiting or refusing emergency or post-stabilization services for enrollees without 
cost and allowing the attending physician to determine stabilization for discharge or transfer 
(438.114). Several instances of questionable language were identified in Member Handbooks. In one 
Member Handbook (Mercy Health Plan), there was language indicating that the member could go to 
an OB/GYN after receiving a referral from the PCP. Later in the Handbook, there was language 
explaining that the member could go to the women’s health provider of their choice, but the 
language did not clarify the original statement implying that the member would need a referral to go 
to an OB/GYN (see 438.206(b)(2)]. Another Member Handbook (Blue Advantage Plus) included a 
statement that a member must have prior approval to seek prenatal care. This distinction could 
easily discourage a member from seeking prenatal care, particularly in the early stages of pregnancy. 
One MC+ MCO (Mercy Health Plan) acknowledged that they did not have nurse midwives in their 
network and were not actively recruiting this resource. They understood that they were obligated 


to provide this service, but it was not available in the current network. 


All MC+ MCOs struggled with having an adequate network in some specific areas of healthcare 
specialization. Five of the seven MC+ MCOs (71.4%) Met the criteria for ensuring an adequate 
provider network [438.206(b)(1)(i-v)]; and three (42.9%) Met regulations for timely access to care 
[438.206(c)(1)(i-vi)], and direct access to specialists [438.208(c)(4)]. The most prevalent problem 
was in the area of orthopedic surgery. MC+ MCO staff explained that the orthopedic surgeons as a 
group, made the decision that they would not contract to serve Medicaid patients due to the low 
reimbursement rate. Each MC+ MCO was able to explain the actions taken to overcome this 
dilemma. A number of the MC+ MCOs (Mercy Health Plan, HealthCare USA, Missouri Care) 


partnered with the teaching hospitals in their Regions, which increased the available surgical 
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capacity. Others (Blue Advantage Plus, Mercy Health Plan) have been able to engage specialists 
through their commercial network and have paid these providers at the commercial rather than the 
Medicaid rate. The MC+ MCOs did not believe that MC+ Members went without needed 
orthopedic services, but these services have been more difficult to obtain. The MC+ MCOs 
reported that the same situation was true for neurosurgery, but this specialty was required less 


often. 
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Table 44. Subpart D: Quality Assessment and Performance Improvement: Access Standards. 
All MC+ MCOs 





Number 
: Partially |Number| Rate 
Federal Regulation CCP HCUSA MoCare| Met Not Met! Met 


438.206(b)(1)(i-v) Availability of Services: Provider 


Network 71.4% 
438.206 (b) (2) Access to Well Woman Care: Direct 





Access 1 0 2 2 2 2 1 4 1 57.1% 
438.206(b)(3) Second Opinions 2 2 2 2 2 2 2 7 0 0 100.0% 
438.206(b)(4) Out of Network Services: Adequate and 

Timely Coverage 1 2 1 2 1 2 2 4 3 0 57.1% 
438.206(b)(5) Out of Network Services: Cost Sharing 1 1 1 2 2 2 1 3 4 0 42.9% 
438.206(c)(1)(i-vi) Timely Access 1 1 iI 2 2 1 2 3 4 0 42.9% 
438.206(c)(2) Provider Services: Cultural Competency 2 2 2 2 2 2 2 7 0 0 100.0% 
438.208(b) Care Coordination: Primary Care 1 2 2 2 1 2 2 5 2 0 71.4% 
438.208(c)(1) Care Coordination: Identification 2 2 1 2 2 2 2 6 1 0 85.7% 
438.208(c)(2) Care Coordination: Assessment 1 1 2 1 2 2 2 4 3 0 57.1% 
438.208(c)(3) Care Coordination: Treatment Plans 1 1 1 2 2 2 2 4 3 0 57.1% 
438.208(c)(4) Care Coordination: Direct Access to 

Specialists 0 1 2 2 1 2 1 3 3 1 42.9% 
438.210(b) Authorization of Services 0 1 1 1 1 2 2 2 4 1 28.6% 
438.210(c) Notice of Adverse Action 1 1 1 1 2 2 2 3 4 0 42.9% 
438.210(d) Timeframes for Decisions, Expedited 

Authorizations 1 1 2 1 2 2 2 4 3 0 57.1% 
438.210(e) Compensation of Utilization Management 

Activities 1 2 2 2 2 2 2 6 1 0 85.7% 
438.114 Emergency and Post-Stabilization Services 0 0 1 1 1 2 2 2 3 2 28.6% 
Number Met 4 6 9 12 12 15 14 72 42 60.5% 
Number Partially Met 10 9 8 5 5 2 3 

Number Not Met 3 2 0 0 0 0 0 

Rate Met 23.5% 35.3% 52.9% 70.6% 70.6% 88.2% 82.4% 


Note: 0 = Not Met; |= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient 
Health Plans (PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 
External Quality Review Monitoring MCOs Protocols. 
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QUALITY ASSESSMENT AND PERFORMANCE IMPROVEMENT: STRUCTURE AND 
OPERATION STANDARDS 


There are [0 Structure and Operations Standards for ensuring compliance with State policies and 
procedures for the selection and retention of providers, disenrollment of members, and 
accountability for activities delegated to subcontractors. There were two items across MC+ MCOs 
that were not Partially Met or Met for Structure and Operation Standards [438.214(c) and 438.12; 
and 438.228]. Across MC+ MCOs, 65.7% of the regulations were Met (see Table 45). 


The Provider Services departments of all MC+ MCOs exhibited a sound and thorough 
understanding of the requirements for provider selection, credentialing, nondiscrimination, 
exclusion, and MC+ Managed Care Program requirements. Four of the seven MC+ MCOs (57.1%) 
Met the regulation for provider selection [438.214(a,b)]. For six of the seven (85.7%) MC+ MCOs, 
there were policies that ensured against provider discrimination for those providers serving 
members with high risk or high cost conditions [438.214(c) and 438.12]. In some cases, MC+ 
MCOs (Blue Advantage Plus, Mercy Health Plan, FirstGuard) described credentialing and re- 
credentialing policies that exceeded the requirements of the regulations. Providers were willing to 
submit to these stricter standards to maintain network qualification in both the MC+ and 
commercial networks of these MC+ MCOs. MC+ MCOs Met 30.0% (Mercy Health Plan) to 90.0% 


(FirstGuard, Blue Advantage Plus) of the standards for Structure and Operations. 


The staff at each MC+ MCO understood the requirements for disenrollment of members, with four 
of the seven MC+ MCOs (57.1%) that Met the applicable regulations [438.226 and438.56(b)(I-3)]. 
Several MC+ MCOs (Mercy Health Plan, First Guard) were able to claim a very small proportion of 
members who chose to disenroll due to a problem or complaint about the MC+ MCO. Most 
changes occurred due to members relocating outside of a MC+ Region, or a change in eligibility 
status. All Member Handbooks included sections explaining how members can change MC+ MCOs 


but did not describe the topic of disenrollment. 


All MC+ MCOs had member grievance systems in place, and this area of practice was a strength for 
them. Six of the seven (85.7%) MC+ MCOs Met the regulation for having a grievance system in 


place. In general, the MC+ MCO policies met most federal and MC+ Managed Care Program 
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requirements and respected member rights. In one case (Community Care Plus), the policy was 
outdated, referring to the previous process of complaint, grievances and appeals. This MC+ MCO’s 


policy had not yet been revised or submitted to the SMA for review and approval. 


The MC+ MCOs all understood the required oversight of subcontractors, but one MC+ MCO 
(14.3%; Missouri Care) Met the regulation for oversight and accountability for any functions and 
responsibilities designated to the subcontractor by providing for mechanisms of revoking the 
delegation, imposing sanctions, monitoring performance, and identifying deficiencies for 
improvement [438.230(a,b)]. MC+ MCOs gave examples of issues that derived from member 
complaints. As these were investigated, a number of issues led to corrective actions for the 
subcontractor. In one instance (Family Health Partners), a series of grievances occurred regarding a 
transportation provider that had inadequate car seats. The MC+ MCO and transportation provider 
collaborated to correct this problem. They ensured that every transportation subcontractor had a 
functioning car seat available. Grievances dropped dramatically, and the MC+ MCO believed the 
transportation provider was monitoring the network to ensure that the use of car seats was in 


compliance with accepted child safety practices and Missouri law. 


All deficiencies for Structure and Operation standards related to a lack of developed or submitted 
policies. In some cases (Family Health Partners, Mercy Health Plan) no policies existed and it did 
not appear that there was a firm understanding of the need for a policy. In most instances, the MC+ 
MCOs could describe existing practices that achieved compliance with regulations, but written 
policies were not submitted for review and approval by the SMA. Four MC+ MCOs (Mercy Health 
Plan, HealthCare USA, Community Care Plus, Family Health Partners) conceded that they were 
aware of this gap in compliance with MC+ Managed Care Program requirements or federal 
regulations. The importance of having approved documentation in place was emphasized to the 


MC+ MCOs by the EQRO at exit conferences. 
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Table 45. Subpart D: Quality Assessment and eer ance Improvement: Structure and ne Peraeen eel 





All MCOs 


Rate 
Federal Regulation | Not Met Met 

438.214(a,b) Provider Selection: 
Credentialing/Recredentialing 1 2 4 3 0 57.1% 
438.214(c) and 438.12 Provider Selection: 
Nondiscrimination 2 2 2 1 2 2 2 6 1 0 85.7% 
438.214(d) Provider Selection: Excluded Providers 1 2 2 2 0 2 2 5 a a 71.4% 
438.214(e) Provider Selection: State Requirements 2 1 2 2 1 2 2 5 2 0 71.4% 
438.226 and 438.56(b)(1-3) Disenrollment: Requirements 
and limitations 1 1 1 2 2 2 2 4 3 0 57.1% 
438.56(c) Disenrollment Requested by the Enrollee 2 1 1 2 2 2 2 5 2 0 71.4% 
438.56(d) Disenrollment: Procedures 2 1 1 2 2 2 2 5 2 0 71.4% 
438.56(e) Disenrollment: Timeframes 1 1 2 2 2 2 2 5 2 0 71.4% 
438.228 Grievance System 1 2 2 2 2 2 2 6 1 0 85.7% 
438.230(a,b) Subcontractual Relationships and Delegation 1 0 1 2. 1 il 1 1 5 1 ao cua 
Number Met 4 3 6 9 6 9 9 46 22 2 65.7% 
Number Partially Met 6 6 4 1 3 1 1 
Number Not Met 0 1 0 0 1 0 oO 


Rate Met 0. 0% 30.0% 60.0% 90.0% 60.0% 90.0% 90.0% 





Note: 0 = Not Met; |= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient 
Health Plans (PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 
External Quality Review Monitoring MCOs Protocols. 
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QUALITY ASSESSMENT AND PERFORMANCE IMPROVEMENT: MEASUREMENT AND 
IMPROVEMENT 


There are 12 Measurement and Improvement Standards addressing the selection, dissemination, and 
adherence to practice guidelines; implementation of performance improvement projects, calculation 
of performance measures, evaluation of utilization, and assessment of services for enrollees with 

special healthcare needs; and the maintenance of information systems that can be effectively used to 
examine service utilization, grievances and appeals, and disenrollment. All items were either Met or 
Partially Met for compliance with Measurement and Improvement (see Table 46). A total of 61.0% 


of the criteria were Met by MC+ MCOs. 


During the on-site review, all MC+ MCOs provided information that indicated the implementation 
and promulgation of national standards for practice guidelines. In the Western MC+ Region, 
members of MC+ MCO staff attended a quality enhancement group meeting for members of the 
healthcare community (Kansas City Quality Improvement Consortium). Regional standards and 
practices were discussed and new standards were developed by this organization. If the regional 
standards exceeded those of the national groups, the regional standards were applied. All MC+ 
MCOs related that they expected providers to use the practice guidelines combined with their 
experience and patient knowledge in their decision-making. If conflicts occurred, the Medical 
Director reviewed the situation with the provider in an effort to approve services that were most 
beneficial to the member. MC+ MCOs Met 18.2% (Community Care Plus) to 72.7% (Family Health 
Partners, Blue Advantage Plus, and Mercy Health Plan) of the regulations for Measurement and 


Improvement. 


All MC+ MCOs were using nationally accredited criteria for utilization management decisions 
[438.240(b)(3)]. The tools most commonly used were the InterQual Clinical Decision Support Tool 
and Milliman Care Guidelines. Both sources provided evidence-based criteria and best practice 
guidelines for healthcare decision-making. MC+ MCO staff discussed how they utilized these tools 
and applied them to member healthcare management issues. Utilization management staff stated 
that they used all the information they had available that fostered evidence-based practice and 


ensured member safety while controlling medically unnecessary care. 
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The MC+ MCOs were actively involved in developing and improving their Quality Assessment and 
Improvement Programs. Several of the MC+ MCOs (Missouri Care, Blue Advantage Plus, First 
Guard, Mercy Health Plan, Health Care USA) convened internal groups that reviewed policies, 
practices, grievances and appeals. This information was used to identify problem areas and to 
improve and enhance their programs. Several MC+ MCOs (Mercy Health Plan, Family Health 
Partners, and FirstGuard) were also in the process of incorporating members into the quality 
improvement process. One MC+ MCO (Family Health Partners) used members to review and 
recommend improvements of their written member materials. Two MC+ MCOs (Mercy Health 
Plan, FirstGuard) were recruiting members to become part of member advisory committees to 


ensure that their perspective was included in decisions about changing and enhancing programs. 


Several problem areas that led to deficiencies in this portion of the protocol involved the Validating 
Performance Improvement Projects, Validating Performance Measures, Validating Encounter Data, 
and Health Information Systems(HIS). Detailed findings and conclusions for these items are 


provided in previous sections of the report and MC+ MCO summaries. 


Performance Management Solutions Group [37 
8. HC a Division of Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 





Table 46. Subpart D: Quality Assessment and Performance Improvement: Measurement and Improvement. 





All MC+ MCOs 


Number 
Partially Number | Rate 
P | | BA+ Met |NotMet| Met 
2 


Federal Regulation 





exam Meey.\ 
1 2 2 










438.236(b)(1-4) Practice Guidelines: Adoption 2 2 2 6 1 0 85.7% 
438.236(c) Practice Guidelines: Dissemination 1 2 2 2 2 2 2 6 1 0 85.7% 
438.236(d) Practice Guidelines: Application 2 2 1 2 2 2 2 6 1 0 85.7% 
438.240(a)(1) QAPI: General Rules 1 2 2 2 2 2 2 6 1 0 85.7% 
438.240(b)(1) and 438.240(d) QAPI: Basic Elements of 

MCO Quality |mprovement and PIPs 1 2 1 1 1 1 2 2 5 0 28.6% 
438.240(b)(2)(c) and 438.204(c) QAPI: Performance 

Measurement a! al 2 a} 2 1 1 2 5 0 28.6% 
438.240(b)(3) QAPI: Basic Elements/Over and Under 

Utilization 1 2 2 2 2 2 2 6 1 0 85.7% 
438.240(b)(4) QAPI: Basic Elements regarding Special 

Healthcare Needs 1 2 2 2 2 2 2 6 1 0 85.7% 
438.240(e) QAPI: Program Review by State NA NA NA NA NA NA NA NA NA NA NA 
438.242(a) Health Information Systems 2 2 2 2 2 2 Z 7 0 0 100.0% 
438.242(b)(1,2) Health Information Systems: Basic 

Elements 1 1 1 1 1 1 1 0 7 0 0.0% 
438.242(b)(3) Health Information Systems: Basic 

Elements 1 1 1 1 1 1 1 0 7 0 0.0% 
Number Met 2 8 7 7 8 7 8 

Number Partially Met 9 3 4 4 3 4 3 

Number Not Met 0 0 0 0 0 0 0 






Rate Met 18.2% 72.7% 63.6% 63.6% 72.7% 63.6% 72.7% 
Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least annually, the impact and effectiveness of each MCO's 
quality assessment and performance improvement program. The regulation refers to the state QA & | program review process and is not applicable to External Quality Review 
of the MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed Care Program. 

0 = Not Met; |= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient 
Health Plans (PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 
External Quality Review Monitoring MCOs Protocols. 
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GRIEVANCE SYSTEMS 

Subpart F of the regulatory provisions for Medicaid managed care (Grievances and Appeals) sets 
forth 18 requirements for notice of action in specific language and format requirements and 
communication to providers and subcontractors regarding grievance and appeal procedures and 
timelines available to enrollees and providers. Six of the seven MC+ MCOs (85.7%) excelled in their 
compliance with the regulations related to grievances and appeals (see Table 47). A total of 91.3% 
of the items Met compliance with regulations for Grievance Systems. Four MC+ MCOs (Mercy 
Health Plan, HealthCare USA, Family Health Partners, and Blue Advantage Plus) were fully compliant 


with the regulations, while one MC+ MCO (Community Care Plus) Met 50.0% of the requirements. 


Grievance and appeal records reviewed for members and providers indicated that the MC+ MCOs 
did not discourage this process, and that they took timely actions when grievances or appeals were 
filed. MC+ MCO staff understood their role in assisting members when they experienced problems 
in completing paperwork or in understanding the grievance or appeal process. Notices contained 
appropriate language, and informed members of their right to request a State Fair Hearing if an 
appeal was negative, or concurrently if the member wished for the action to be reviewed at the 
State level while the MC+ MCO was taking action. Member Handbooks and notices informed 
members of their right to retain their benefits while an appeal was pending, but also notified them of 
their responsibility to reimburse providers for services, if an appeal or State Fair Hearing upheld the 
actions taken by the MC+ MCO. Records of appeals filed by providers demonstrated that they 
were comfortable filing on behalf of a member. Providers were informed in their policy that they 


must have the member’s written permission to file on their behalf. 


There were several deficiencies in the area of policy submission to the SMA. Two MC+ MCOs 
Partially Met compliance with regulations for establishing and maintaining an expedited review 
process for appeals (Community Care Plus, Missouri Care; 438.410); and two Partially Met 
compliance with regulations for providing information regarding the grievance and appeal process to 
all subcontractors at the time of contracting (Community Care Plus, FirstGuard; 438.414). One 
MC+ MCO (Community Care Plus) did not have updated policies reflecting the current language 
and structure of the grievance and appeal process. Appropriate practice for addressing grievances 


and appeals appeared to be in place for all MC+ MCOs. 
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Table 47. Subpart F: Grievance Systems. 
All MC+ MCOs 


Number 
Partially | Number 
| MOCare Met Not Met | Rate Met 


MC+ I MCO — 





Federal Regulation 
















438.402(a) Grievance and Appeals: General Requirements 2 2 2 2 2 Z qi 0 100.0% 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 2 2 2 2 2 7 ; 0 100.0% 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 2 2 2 2 2 7 0 0 100.0% 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 2 2 2 2 2 7 0 0 100.0% 
438.404(a) Grievance System: Notice of Action - Language and Format il 2 2 2 2 Z 2 6 I 0 85.7% 
438.404(b) Notice of Action: Content Ad 2 2 2 2 2 2 6 di 0 85.7% 
438.404(c) Notice of Action: Timing 1 2 2 2 2 2 Z 6 il 0 85.7% 
438.406(a) Handling of Grievances and Appeals: General Requirements 1 2 2 2 2 2 2 6 1 0 85.7% 
438.406(b) Handling of Grievance and Appeals: Special Requirements for 

Appeals 2 2 2 2 2 2 2 7 0 0 100.0% 
438.408(a) Resolution and Notification: Basic Rule 2 2 2 2 2 2 2 7 0 0 100.0% 
438.408(b,c) Resolution and Notification: Grievances and Appeals - 

Timeframes and Extensions 2 2 2 2 2 2 2 7 0 0 100.0% 
438.408(d)(e) Resolution and Notification: Grievance and Appeals - Format 

and Content of Notice 1 2 2 2 2 2 2 6 I 0 85.7% 
438.408(f) Resolution and Notification: Grievances and Appeals - 

Requirements for State Fair Hearings il 2 2 2 2 2 2 6 il 0 85.7% 
438.410 Expedited Resolution of Appeals 1 2 2 1 2 2 2 5 2 0 71.4% 
438.414 Information about the Grievance System to Providers and 

Subcontractors ih 2 2 2 2 il 2 5 2 0 71.4% 
438.416 Recordkeeping and Reporting Requirements 0 2 2 2 2 2 2 6 0 I 85.7% 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 2 2 2 2 2 a 0 0 100.0% 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 2 2 2 2 2 7 0 0 100.0% 
Number Met 9 18 18 17 18 17 18 1 

Number Partially Met 8 0 0 1 0 1 0 

Number Not Met 1 





Rate Met 
Note: 0 = Not Met; |= Partially Met ; 2 = Met 





50.0% 


100.0% 


100.0% 


94.4% 


100.0% 


94.4% 






100.0% 


Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient 
Health Plans (PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 


External Quality Review Monitoring MCOs Protocols. 
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Table 48 provides a summary of the compliance ratings across all sections of the Compliance 
Protocol. All MC+MCOs Met 68.3% of the applicable regulations, with individual MC+ MCOs 
meeting 30.4% (Community Care Plus) to 88.4% (FirstGuard) of all applicable regulations. Four 
MC+ MCOs (HealthCare USA, Missouri Care, FirstGuard and Blue Advantage Plus) Met or Partially 


Met all regulations. 


Table 48. Summary of MC+ MCO Compliance with Medicaid Managed Care Regulations. 


MC+ MCO 


Federal runt cp (ap HCUSA a eee 
21 37 45 52 54 61 


Number Met 

Number Partially Met 42 17 14 8 
Number Not Met 0 

Number Applicable 69 


Percent Met 53.6% 65.2% 715.4% 78.3% 

Note: 0 = Not Met; |= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid 
Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for determining compliance with 
Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 External 
Quality Review Monitoring MCOs Protocols. 





Follow-Up 


The EQRO conducted follow-up of recommendations based on the 2003 EQRO Report findings 


submitted to the SMA on August 31, 2004 that related to the Compliance Protocol. 


|. The findings of the 2003 EQRO evaluation revealed rates lower than the national average for 
follow-up after psychiatric hospitalization at seven and thirty days, based on Mental Health 
Indicator data. Review of case management records found the presence of a treatment plan in 
29% of the records reviewed, with 24% containing objective and measurable treatment goals. 
Referrals comprised the primary case management activity. Review of behavioral health case 
management processes to determine compliance with federal guidelines related to members 
with special healthcare needs, access to care, and ensuring appropriate access to and follow-up 
of care was recommended for MC+ MCOs. MC+ MCOs implemented case management to 
improve follow-up contact of members and observed a reduced frequency of re-admission rates, 
as well as an increase in aftercare appointments. There were no specific initiatives focused on 
improving the documentation of case management and change in 7- and 30-day follow-up rates. 
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2. Findings of the 2003 EQRO evaluation indicated MC+ MCO needs for understanding sampling, 
study question development, statistical analysis, and the implementation of PIPs. 
Recommendations were directed toward MC+ MCOs for the inclusion of all QI/UM staff in PIP 
implementation and study design, the targeting of resources on PIP implementation, and training 
for staff. MC+ MCOs were referred to the CMS Protocols for conducting and evaluating PIPs. 
At the time of the 2004 EQRO review, and after training conducted by the EQRO specific to 
the implementation of PIPs, personnel at MC+ MCOs remained unaware of the PIP Protocols 
and requirements. 


3. Results of a childhood lead poisoning evaluation focused study, conducted for the 2003 EQRO, 
found that although there was continued improvement in the rates of blood lead level testing for 
children 12 and 24 months of age, the rates were low (51.5% and 38.7%, respectively). The rate 
for 24-month-olds was particularly low. The HCY Lead Risk Assessment Guide (LRA), the 
verbal lead screen items, was administered 73.9% of the time to MC+ MCO members six year 
of age and under. Only 10.6% of MC+ Members with a positive response on the LRA received a 
blood lead level test within three months of the positive response. Recommendations were 
made for improving the follow-up of verbal lead screens with a capillary sample or blood lead 
level test, improving the rate of 24-month-olds’ blood lead level tests, and an incorporation of 
verbal lead screening items into the baseline health assessment at enrollment and for MC+ 
MCO new member welcome calls. The MC+ MCOs continued case management, education, 
and follow-up services for members with lead exposure. Monitoring of the rate of blood lead 
levels for |2- and 24-month-olds also continued, but there were no concerted efforts to 
improve follow-up for members under six years of age with a positive verbal lead screen. 


4. The provider network adequacy study conducted for the 2003 EQRO evaluation year found 
that MC+ MCO submissions to the State Department of Insurance (Missouri Department of 
Insurance, MDI) contained duplicate entries of providers and showed high rates of closed 
provider panels. Additionally, the MDI submissions no longer addressed dental and behavioral 
health provider network adequacy. To ensure adequate access to provider networks, it was 
recommended that the SMA and MC+ MCOs identify and monitor several indices for provider 
network adequacy in addition to the continued monitoring of after hours, emergency, and next 
appointment availability standards for individual providers. Quarterly evaluation of the adequacy 
of dental and behavioral health providers using net gain/loss indices by provider specialty, and 
index (proportion) of closed panels, and Provider:Member ratios was recommended. Additional 
indicators of provider adequacy that are collected for other purposes (e.g., member satisfaction) 
and were recommended as proxies of provider availability were access to care indicators (well- 
care visits, annual dental visits, follow-up after psychiatric hospitalization). MC+ MCOs 
continued to conduct provider after-hours access monitoring, and coordinated referrals to out- 
of-network providers, but there were no efforts to develop and review specific indicators for 
ensuring that members who need specialized care (e.g. dental, behavioral health, orthopedic) are 
receiving it in a timely manner. 
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Conclusions 


STRENGTHS 


Four MC+ MCOs Met or Partially Met all applicable federal regulations and State compliance 
requirements. 


Ten of the I3 regulations for Enrollee Rights and Protections were Met or Partially Met by MC+ 
MCOs. Communicating MC+ Members rights to respect, privacy, and treatment options as well 
as providing written information in prevalent non-English languages were areas of strength. 

MC+ MCOs maintained an awareness of and appropriate responses to cultural and language 
barriers to communication and obtaining care. The MC+ MCOs responded to barriers, 
whether physical, emotional, or cultural with diligence and creativity. 


Another area of strength related to Enrollee Rights and Protections was the provision of 
information to MC+ Members regarding advanced directives, grievance and appeals processes, 
and potential liability for payment. 


Twelve of the |7 regulations for Access Standards were Met or Partially Met by MC+ MCOs. 
MC+ MCOs Met regulations for having in place policies and procedures to provide for second, 
and if necessary, third opinions in meeting the healthcare needs of MC+ Members. Access to 
culturally competent care was also a strength. MC+ MCOs monitored high risk MC+ Members 
by employing specialized nurses who had regular contact with the members or their families to 
perform case management. At each MC+ MCO, procedures were described for obtaining the 
identities of members who had special health care needs from the SMA on a monthly basis. 
Additionally, MC+ Members in need of case management were identified and included in case 
management services by the MC+ MCOs. Members immediately assessed for case management 
services included children who were: 


e —_ Eligible for Supplemental Security Income (SSI); 
e In foster care or other out-of-home placement; 
e Receiving foster care or adoption subsidy; and 


e Receiving services through a family-centered community-based coordinated care system that receives grant 
funds under Section 501 (a)(1)(D) of Title V, as defined by the state agency in terms of either program 
participant or special health care needs. 


Additionally, MC+ MCOs attempted to identify members with a number of high risk conditions 
that included: 
e Lead poisoning; 


e Prenatal risk factors; 
e Behavioral health needs: or 
e Other chronic medical conditions. 


Ten of the |2 regulations for Structure and Operations Standards were Met or Partially Met by 
MC+ MCOs, as there were mechanisms in place to avoid discrimination of providers specializing 
in treating MC+ Members with high risk or high cost conditions, and grievance systems. The 
MC+ MCOs made positive changes in their oversight and involvement with the behavioral 
health contractors. Some were in the process of integrating delivery of mental and behavioral 
health services into the function of the MC+ MCO. The MC+ MCOs acknowledged that 


Performance Management Solutions Group 163 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


theprovision of meaningful mental health services complemented and strengthened their efforts 
to provide sound physical healthcare initiatives for many MC+ members. M 


All | | regulations for Measurement and Improvement were Met or Partially Met by MC+ 
MCOs. MC+ MCOs adopted, disseminated, and applied practice guidelines to measure the 
improvement in quality of care and health outcomes. They also used health information systems 
to examine the appropriate utilization of care using national standard guidelines for utilization 
management. MC+ MCOs incorporated methods of quality assessment and program 
improvement into the daily operations of their organizations. Several MC+ MCOs started 
incorporating member suggestions into program improvement, planning and development. 

Their grievance and appeals systems were used to inform and drive improvements in practice. 
Improvements were found in the adoption and implementation of nationally accepted clinical 
practice guidelines with additional Regional standards that exceeded the national guidelines. The 
Provider Services or Relations departments of the MC+ MCOs exhibited a commitment to 
relationship building and monitoring providers to ensure that all standards of care were met and 
that good decision-making and sound healthcare practices occurred on behalf of MC+ Members. 


Seventeen of 18 regulations for Grievance Systems were Met or Partially Met by MC+ MCOs. 
Nine of the 18 were Met by all seven MC+ MCOs, with strong systems in place for the 
management of member grievances. 


MC+ MCOs were invested in developing programs and providing services beyond the scope of 
contractual requirements. Partnerships with the local universities and medical schools provided 
opportunities to obtain cutting-edge and occasionally experimental treatment options, which 
would not otherwise be available to MC+ Members. 


AREAS FOR IMPROVEMENT 


MC+ MCOs did not submit Member Handbooks, Marketing and Educational Materials, and 
dissemination plans to the SMA for review and approval. Given that these are primary modes of 
communication with MC+ Members for rights and protections, a number of regulations were 
Partially Met or Not Met. These included providing MC+ Members with assurances that they 
are not liable for MCO insolvency or services not paid for by the SMA, or for fees in excess of 
those paid by the MCO that are furnished under contract, referral, or other arrangement. 


MC+ MCOQOs did not all have written policies to ensure the consistent application of review 
criteria for the authorization of services or for addressing the need for access to emergency and 
post-stabilization services. 


There was a lack of understanding by some MC+ MCOs of the need for written policies 
addressing subcontractor oversight and delegation, disenrollment, and credentialing and re- 
credentialing. Although process may have been in place, specific policies were not developed or 
updated and submitted to the SMA for review and approval. 


The use of data for quality improvement purposes and examination of healthcare outcomes or 
processes strongly associated with healthcare outcomes is an area in need of improvement. 
MC+ MCOs collect monitoring data for various requirements, but do not compile or evaluate 
the findings in a way to address specific clinical or nonclinical issues related to quality and access 
to care. 
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The MC+ MCOs continue to struggle with regular access to certain types of specialist care. 
Throughout discussions at each MC+ MCO, the lack of orthopedic surgeons, neurosurgeons 
and child psychiatrists was mentioned as a problem. The MC+ MCOs have all made 
accommodations to ensure that members received the services they required. Through the use 
of advance practice nurses, cooperative agreements with medical schools, and willingness to 
reimburse at commercial insurance rates, the MC+ MCOs attempt to ensure that members 
have access to these services. MC+ MCOs expressed concern for improvement in this area. 


The MC+ MCOs could improve their local understanding of the data and information in the 
health information systems, and what is required to extract this data and information. The MC+ 
MCOs maintained a hands-off attitude regarding their health information systems. In some 
cases, there was very little local expertise about what is contained in these systems, and how to 
make pertinent requests for information when data extraction is necessary. The information 
collected in these systems could be used to inform and strengthen performance improvement 
and program development efforts. 
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RECOMMENDATIONS 


|. As part of the process for conducting the review of MC+ MCO compliance with federal 
regulations for Medicaid managed care, the EQRO developed a tool to conduct the Compliance 
Protocol based on the SMA contract compliance findings and EQRO on-site reviews. This tool 
was completed for each MC+ MCO, with detailed comments for each regulation that was not 
Met. The findings for each MC+ MCO were provided to the SMA. It is recommended that the 
completed tools be distributed to MC+ MCOs to aid in improving compliance with specific 
regulations. 


2. One reason that MC+ MCO compliance with federal regulations was Partially Met or Not Met 
was the lack of development and/or submission of established policies to the SMA for review 
and approval. It is recommended that MC+ MCOs prioritize those documents used for MC+ 
Member communication (e.g., Member Handbooks and Marketing and Education Materials) for 
revision, internal approval, and submission to the SMA. 


3. It is recommended that MC+ MCOs finalize and submit any pending policies relating to review 
criteria for authorization of services and access to emergency and post-stabilization services. 


4. To address provider network timeliness, access and availability issues, it is recommended that 
MC+ MCOs review the regulations, revise policies and procedures for SMA review and 
approval, and develop methods of monitoring how well the needs of members for specialists are 
being met. As recommended in the 2003 EQRO Report, MC+ MCOs should be evaluating the 
monitoring data collected from several sources on at least a quarterly basis to ensure that MC+ 
Members are not encountering barriers in accessing needed specialty services due to closed 
panels or specialty provider unavailability within the network. Specialties for which MC+ MCOs 
were granted exceptions through the Missouri Department of Insurance geographic network 
adequacy analysis would be appropriate targets, in addition to the specialties no longer included 
in this process (dental and behavioral health) and those identified by MC+ MCOs as difficult or 
impossible to engage (neurologists, orthopedic specialists). Surveys of MC+ Members with 
specific diagnoses suggesting the need for specific specialties can also be conducted to determine 
the extent to which MC+ Members feel their needs for specialty care are met, and the barriers, 
if any, they encounter in accessing specialty care. The National Survey of Children with Special 
Health Care Needs?’ may provide some ideas for items that may be adapted for such a purpose. 


5. MC+MCQOs should review regulations and contract requirements related to subcontractual 
relationships and delegation. Specific policies and procedures should be developed and 
submitted to the SMA for review and approval. It is recommended that MC+ MCOs review 
regulations on credentialing and re-credentialing of providers and disenrollment of members, 
revise policies as necessary and review them with staff. 


6. Specific recommendations for conducting Performance Improvement Projects that result in 
significant, sustained improvement over time; using health information systems to calculate 
performance measures; and providing encounter data to the SMA and EQRO for validation 
purposes are provided in previous sections of this report. 


7. MC+MCOs should develop a system to monitor the need for revision of internal policies and 
procedures, review and update annually or when contract revisions occur (whichever comes 
first), and ensure submission of policy changes to the SMA within appropriate timeframes. 


*”7 Centers for Disease Control (2002). SLAITS National Survey of Children with Special Health Care Needs, 
retrieved May 31, 2005, http://cshcndata.org/anonymous/documents/survey. pdf. 
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COMMUNITY CARE PLUS 





The previous sections of the 2004 EQRO report present the purpose and objectives technical 
methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 


Document Review 
Community Care Plus supplied the following documentation for review: 


e Performance Improvement Project: 2004: Twenty-Four Hour Provider Access Monitor 
e Performance Improvement Project 2004: Emergency Room Utilization 


Interviews 

Interviews were conducted with the project leaders for each of the projects on-site by the EQRO 
Project Director, on Tuesday, March |, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided as were references for logic 
model development and health services research methods. The following questions were addressed 
on-site: 


Who was the Project Leader? 

How was the topic identified? 

How was the study question determined? 

What were the findings? 

What was the intervention? 

What was the time period of the study? 

Was the intervention effective? 

How do you know? 

Why or why not? 

What does Community Care Plus want to study or learn from their PIPs? 
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FINDINGS 

The first PIP evaluated was “Emergency Room Utilization”. This project “is a program designed to 
decrease inappropriate use of the emergency room, contain costs and to provide education to 
providers and members.” The project involved daily utilization management and phone calls and 
letters to members. There was a reference to, but no explanation of the role of educating 
providers about the referral process. There were no study questions presented, but the implied 
question might be, “Does education of members and providers about the importance of routine 
office visits reduce inappropriate use of the emergency room and contain cost?” 

The demographic characteristics of members were not described, but included all members 
attending one pediatric facility. It was not clear whether the study population included all MC+ 


Members. No sampling was conducted. 


The selected study indicators consisted of the number of emergency room visits, number of 
members receiving telephonic education, number of members receiving letters regarding Nurse 
Line, and the number of regularly scheduled office visits with a PCP. There was no measure of the 
number of inappropriate emergency room visits. The measures were not clearly defined and no 
benchmarks or baseline data were presented. It was unclear how the indicators were going to be 
evaluated or what would indicate success in the intervention. There were no measures for change 
in health status or processes strongly associated with health status. Emergency room utilization 
data were collected for all members included in the study. The requirements for data collection 
procedures were not met, as there was no information about measures and methods for calculation. 
There was no specific survey tool, but there was a flowchart for conducting telephone calls. The 
emergency room follow-up flowchart suggested possible process data to be collected. Process data 
on the ability to contact members and the number of phone calls and letters were provided, as well 
as were graphs of time periods for high ER utilization and patterns of ER utilization during the week. 
However, the connection between this information and the measurement of emergency room use 


was not made. 


There was no data analysis plan identified in the narrative, so it was not possible to evaluate the 
findings against the data analysis plan. Also, the definition of inappropriate utilization was made by 
nurse judgment on a case-by-case basis for members that were not admitted to the hospital. The 
characteristics and a definition of inappropriate utilization should be presented and described. The 


analysis of PIP findings should include clearly labeled figures and graphs as well. Finally, pre- and 
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post-intervention analysis would be important in determining the effectiveness of the intervention. 
For example, “Did the rate of inappropriate department use actually decline following the 


intervention or not?” 


This study is not likely to result in credible or interpretable findings. This project was focused on 
utilization and cost alone, which is not considered appropriate as a PIP. There was good 
measurement of the implementation of the intervention. A number of process measures regarding 
the intervention were presented, but there was no clear link between the education of members 
and the outcome. It would be critical to operationally define and measure “inappropriate 
emergency room use,’ as defined by medical staff. Descriptive data regarding the characteristics of 
individuals and referrals for those with “inappropriate emergency room use’ might suggest a focus 


for additional interventions. 


A study which examines the impact of an intervention on patient outcomes or processes of care 
that are highly associated with patient outcomes rather than utilization or cost alone is necessary. It 
is recommended that a study which examines the reasons for self-referral to the emergency room 
or the impact of the intervention of case management on health outcomes and access to preventive 
care be considered. While collecting data from members about the reason for self-referral, 
questions about accessing care and the potential viability of urgent care options could be explored 
and analyzed as potential barriers for future intervention. The reasons or symptoms for which 
people present to the emergency room may also be useful for identifying unmet needs. When 
calculating measures, examine the change in the rate of the measures prior to and following the 


intervention to determine effectiveness of the intervention. 


The second PIP evaluated was the “Twenty-Four Hour Access Monitor’. Data showing high levels 
of member satisfaction were presented as a rationale for monitoring provider accessibility, but there 
was no evidence of a specific problem other than the need to meet contract compliance 


requirements. 


The study questions stated at the end of the narrative were: |) “Does the PCP provide coverage for 
the member 24/72”; 2) “Are members being told to go to the ER without options provided?”; 3) 
‘Are members going to the ER instead of checking with the PCP or do the members go to the ER 


knowing that the PCP is not available?”; 4) “How does this affect Community Care Plus’s ER 
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utilization cost?”; 5) “Should there be a co-pay for ER to control utilization?”’; 6) “Could utilization 
have improved if there had been urgent care centers?”; 7) “How does this issue affect our 
membership?”; and 8) “Have we lost members?” It was not clear how the measures presented 
were related to provider network accessibility, nor was it possible to identify interventions aimed at 
improving the provider network accessibility other than monitoring and reporting. There was no 
description of the members or providers that were surveyed on the satisfaction measure. It was 
assumed that the Consumer Assessment of Health Plans survey (CAHPS) included only Community 
Care Plus and MC+ Members, but this should be clearly stated. It was not clear how many 
providers, offices, and/or what types of providers were called for the 24-hour access monitoring 
process. The calculation of one of the many measures was described (the portion of providers that 
met “after hours” requirement). Other measures for which data were reported to be collected 
included appointment availability, office waiting times, responsiveness to calls, number of members 
calling providers for help, members with an illness or injury that needed help right away, members 
that did not need to go to the ER in the past 12 months, and members who got help when calling 
the doctor’s office after hours. It is likely that there is a link between member satisfaction and 
provider network adequacy, but it was unclear what intervention or outcomes were being 
measured. There were no demographic characteristics of members that were sampled described, 
or members who responded to the satisfaction survey. The reader knowledgeable of CAHPS 
methodology would be aware that the sampling methods are random. However, this should be 


described. 


There was a description of the criteria for inclusion of providers in the after hours access 
monitoring measure (i.e., the number of PCPs in the network who submitted a claim during the 
measurement year). However, “high volume” providers were subsequently described as being 
targeted, but in the analysis there is an indication that “all PCPs” were surveyed. The definition of 
PCPs (e.g., internist, OB/GYN, mental health), high-volume providers was also unclear. There were 
no sampling techniques, response rates, or non-response biases described for the member 


satisfaction survey. It was assumed there was no sampling for the provider network access monitor. 


The sources of data (member satisfaction and provider office availability) were clear. There was no 
data analysis plan described in the narrative. In terms of the intervention, there appeared to be 
some interventions for non-compliant providers, but there were no causes or barriers to access 


issues or member satisfaction problems identified. In the analysis, raw data for the number of 
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providers meeting after-hours availability were presented for 2002 and 2003. There was no time 
period reported for re-measurement or indication of the number of PCPs that were contacted in 
2003, nor was there discussion of the effectiveness of the intervention. It is possible that the same 
source of data for the provider after-hours monitoring was used from year to year, although there 
was some sampling in 2002, so the method of data collection seems to have varied. It appears that a 
new tool was developed for the 2004 data collection. Variation in data collection, sampling, and 
methods from year to year should be described and taken into account when interpreting data. It 


was not possible to identify whether improvement had occurred as a result of the intervention. 


This study is not likely to result in credible or valid findings. This was a monitoring project using 
indicators of required compliance items for the State and MCO contracts for MC+ as the main 
indicator. Although monitoring is an important aspect of program compliance and process 
evaluation, it does not constitute a targeted activity designed to improve outcomes. In the study 
described, there was no connection between the data, measures, methods, and study questions 
presented. It was unclear what interventions might have taken place to improve the quality, access, 
timeliness or outcomes of care. Although the measures and indices proposed may be useful for 
answering specific questions, it is important to identify indices, methods, measures, and analyses that 
will allow one to answer the study questions posed. The satisfaction measures used indicate a high 
level of satisfaction of members with providers and the data presented show higher than the 


national average levels of member satisfaction, so it is not clear how or why this topic was chosen. 


It is recommended that a topic that has potential for significant improvement in the processes 
(access or timeliness) or outcomes (health) of care be identified. This will require establishing a 
rationale and presenting data describing the necessity of the study and scope of the problem. A PIP 
should be designed to answer specific questions, which should be presented in the narrative, and 
those questions should be directly associated with the topic of the study, measurements, and 


outcomes. 


STRENGTHS 
|. PIP topics were concerned with nonclinical issues such as utilization and access. 


2. There was some description of the study rationale and measures used, and some background 
information for the purpose of the studies. 


3. Measures and their calculation were well-defined. 
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Satisfaction data from members was incorporated into measures. The Consumer Assessment of 
Health Plans Survey (CAHPS) findings were used as background data. 


There were good measures of the implementation of the interventions. The findings 
summarized the process of the implementation, describing the numbers of phone calls made to 
providers and members. 


Since the site visit, Community Care Plus has instituted personnel changes to focus on and 
improve the implementation of PIPs. 


AREAS FOR IMPROVEMENT 


The focus of PIP studies was primarily on utilization, cost, and monitoring of compliance. 
Performance Improvement Projects need to focus on interventions that improve the processes 
and outcomes of care over and above utilization or cost alone. Although measures used for 
monitoring compliance can be useful as measures of process or outcome for PIPs, there must 
also be interventions for improving the process or outcome of care for a broad portion of the 
population served. 


2. Study questions, measures, and analyses were not related to one another. At least quarterly 
data analysis should be conducted to monitor the outcomes. 

3. Measures of the implementation of the intervention (e.g., contacting members or providers) 
should be evaluated continuously. 

4. Data analysis should incorporate tests of statistical significance to assess whether the resulting 
change or lack of change was related to the intervention. 

5. Interventions were not likely strong enough to produce changes in the indices measured. 

RECOMMENDATIONS 

|. The study design of PIPs need to link the question(s), the intervention(s), and the outcomes to 
determine whether or not an intervention is effective. This can be accomplished by developing 
a logic model for the PIP at the planning stage. 

2. Community Care Plus should develop study questions that have measurable results and repeat 
measures at least quarterly. Annual evaluation of measures does not allow for frequent enough 
feedback to adjust interventions or address barriers in a timely manner to improve the quality of 
care. 

3. Include tests of statistical significance to compare pre- and post-intervention changes. Chi- 
Square analysis and upper and lower confidence intervals are useful indices for comparing rates 
over re-measurement periods. 
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Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Community Care Plus. Community Care Plus submitted the requested 
documents on November 23rd, 2004. The EQRO reviewed documentation between November 
23rd, 2004 and February 28th, 2005. On-site review time was used to conduct follow-up questions 


and provide feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by Community Care Plus (prepared by 
Novasys) 

e The Baseline Assessment Tool (BAT) submitted by Community Care Plus (prepared by Novasys) 

e Healthcare Research Associates’ (HRA) HEDIS 2004 Compliance Audit Report 

e NovaSys Health Network, LLC, policies and procedures related to the HEDIS rate calculation process. 

e —NovaSys Health Network, Community Care Plus electronic eligibility process 

e Data files from the HEDIS repository containing eligible population, numerators and denominators for each 
of the three measures 

e Decision rules & queries in the HEDIS 2004 repository used to identify eligible population, numerators and 
denominators for each of the three measures 

e Query result files from the repository 


The following are the data files submitted by Community Care Plus for review by the EQRO: 


e Community Care Plus CLAIM PULL SC.mdb 
e Community Care Plus ADMIN SC.mdb 

e Community Care Plus HEDIS 2004 SC.mdb 
e EQRO TO BHC.mdb 

e HEDIS 2004 COMBINE MRR HRA SC.mdb 


Interviews 
The EQRO conducted on-site interviews with Michael Boone (representing Novasys, the third party 
administrator for Community Care Plus) and Lee Kneibert on Tuesday, March Ist, 2005. Michael 


Boone of Novasys was responsible for calculating the HEDIS 2004 performance measures. 


FINDINGS 

Community Care Plus calculated the Adolescent Immunization Status, Combination #| measure 
using the Hybrid Method. The Adolescent Well-Care Visits and Use of Appropriate Medications for 
Asthma measures were calculated using the Administrative Method. MCO to MCO comparisons of 


the rates of Adolescent Immunization Status Combination #1, Adolescent Well-Care Visits, and Use 
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of Appropriate Medications for People with Asthma measures were conducted using two-tailed z- 
tests. For comparisons that were statistically significant at the 95% confidence interval (Cl), the z- 
score (z), the upper and lower confidence intervals (Cl), and the significance levels (p < .05) are 


reported. 


The rate for the HEDIS 2004 Adolescent Immunization Status, Combination #1 measure reported 
to the SMA and the State Public Health Agency (SPHA) by Community Care Plus was 35.04%. This 
was comparable to the statewide rate for all MC+ MCOs (14.36%; z = .36; 95% Cl: 20.18%, 49.90%; 


n.s.). 


The rate for Community Care Plus for the HEDIS 2004 Adolescent Well-Care Visits measure was 

18.75%, which was significantly lower than the statewide rate for all MC+ MCOs (30.13%; z = -1.43, 
95% Cl: 13.87%, 23.63%; p < .001;). The 2004 HEDIS rate for Community Care Plus for the Use of 
Appropriate Medications for People with Asthma was 53.71%, which was significantly lower than the 


statewide rate for MC+ MCOs (63.92%, z = -1.54; 95% Cl: 50.07%, 57.35%; p < .001). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

Information systems management policies and procedures for rate calculation were evaluated 
consistent with the Validating Performance Measures Protocol. This included both manual and 
automatic processes of information collection, storing, analyzing and reporting. The EQRO was 
provided with a demonstration of the HEDIS repository with the assistance of a remote connection 
from Community Care Plus location in St. Louis to the vendor’s system in Little Rock, Arkansas. 
For the Adolescent Immunization Status, Combination #l and the Use of Appropriate Medications 
for People with Asthma measures, Community Care Plus was found to meet most of the criteria for 
having procedures in place to produce complete and accurate data (see Attachment V: Data 
Integration and Control Findings). There were no biases or errors found in the manner in which 


Community Care Plus transferred data into the repository used for calculating the HEDIS 2004 
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measures. The Adolescent Well-Care Visits measure was not valid, as the prescribed service dates 
were not followed (data from the year prior to the measurement year were counted as 
numerators). This suggests that the report runs during the production process were not adequately 


reviewed for values outside the specified ranges. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate (see Attachment VII: Data 
and Processes Used to Calculate and Report Performance Measures). Community Care Plus met 
nearly all criteria that applied for all three measures. For all three measures, the two criteria that 
were not met involved the use of statistical significance testing to document changes in performance 
over time and using this information to support claims of improvement or stability in performance 
over time. Although many auditors conduct statistical tests to examine various aspects of data 
validity, such tests should be used by the MCO to assess the significance of change related to quality 
improvement activities and operational changes. For the Adolescent Immunization Status, 
Combination #1 measure, the State Public Health Immunization Registry (MOHSAIC) data were not 
incorporated into the calculation of the rates, which may have contributed to an underestimate of 


the actual rate. 


Processes Used to Produce Denominators 

Community Care Plus met all criteria for the processes employed to produce the denominators of 
all three performance measures (see Attachment X: Denominator Validation Findings). This 
involves the selection of members eligible for the services being measured. Four hundred eleven 
(411) sampled members were reported and validated for the Adolescent Immunization Status, 
Combination #| measure. A total of 9,887 members eligible were reported and validated for the 
Adolescent Well-Care Visits measure; and 849 members eligible were reported for the 
denominator of the Use of Appropriate Medications for People with Asthma measure. Age ranges, 
dates of enrollment, medical events, and continuous enrollment criteria were programmed to 
include only those members who met HEDIS 2004 criteria. Member identification numbers and 
dates of birth were within valid ranges for each of the three measures. The dates of enrollment 
represented valid gaps in enrollment and met continuous enrollment requirements. Medical event 
codes were also valid for all three measures. There were no exclusions, contraindications, or 


replacements reported by the MCO. 
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Processes Used to Produce Numerators 

All three measures included the appropriate administrative data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2004 criteria (see 
Attachment XIll: Numerator Validation Findings) for the Adolescent Immunization Status and Use 
of Appropriate Medications for People with Asthma measures. Medical record reviews were 


conducted for the Adolescent Immunization Status measure. 


For the Adolescent Immunization Status measure, Community Care Plus excluded administrative 
events from the State Public Health Immunization Registry (MOHSAIC). There were 143 hits from 
medical record review reported by Community Care Plus. Thirteen (13) of 30 medical records 
requested for review were received and |2 records resulted in validated hybrid hits. One record 
reviewed had neither administrative nor medical record review data for meeting the criteria of 
either two MMRs or three Hep B vaccinations. The medical record review validated 57 of the 143 
hybrid hits reported. The estimated bias from medical record review was 20.88%. Based on the 
number of hits validated by the EQRO, the rate calculated by the EQRO was 14.11%. The total 
estimated bias for the Adolescent Immunization Status, Combination # | measure was 20.92%. This 
bias may be largely accounted for by the low submission rate of records for validation. The EQRO 
worked very closely with Community Care Plus in attempting to educate regarding the measure 


being validated and the need for actual medical record data. 


For the Adolescent Well-Care Visits measure, there were a total of |,873 administrative hits found 
in the data file. Although the dates of birth were found to be within range for the given set of 
patients, the dates of service ranged from 1/2/2002 — |2/31/2002, which was not within the valid 
range of the year 2003. The visits were identified using CPT codes 99383-99385, W0025, 99393- 
99395 and ICD-9CM codes V20.2, V70.0, V70.3, V70.5-6, and V70.8-9. There were |,247 “W0025’ 
codes that are universal codes covering well-care visits for ages from 0-2| years. The final rate 
could not be calculated because of the invalid date of service. The estimated bias with the 
submission in the DST was 18.75%, and the rate reported is not valid. On follow-up, CC Plus 
resubmitted the calculations for this measure and the final rate for this measure was 25.39%, which 


was higher than reported to the SPHA and SMA. 
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The Use of Appropriate Medications for People with Asthma measure was the third measure 
validated. There were a total of 456 administrative numerator events reported and validated. There 
were a total of 818 events that were captured. The diagnosis codes and dates of service were not 
provided. The dates of birth were in the valid range. The dates of service were not provided. The 
dates of enrollment were in the range of 9/1/1998- 12/20/2003 and, as explained by Community Care 
Plus, the dates in 2002-2003 were the re-enrollment dates and primary dates were within valid 


range. The final rate was calculated to be 53.71%, with no bias observed. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Adolescent Immunization Status measure. Attachment XIl; 
Impact of Medical Record Review Findings and Attachment XV: Sampling Validation Findings were 
completed for each of these measures. Community Care Plus employed a 10% oversampling rate 
for a final sample size (FSS) of 453, which is within specified parameters. No exclusions or 


replacements were reported from administrative or medical record review data. 


Submission of Measures to the State 

Community Care Plus submitted the DST for each of the three measures validated to the SPHA 
(the Missouri Department of Health and Senior Services; DHSS) in accordance with the Code of 
State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA 
Quality Improvement Strategy. Upon feedback from the EQRO that the Adolescent Well-Care 
Visits measure was not valid due to the use of 2002 service dates for the numerator, the MCO re- 


submitted corrected findings to SPHA. 


Determination of Validation Findings and Calculation of Bias 

The following table summarizes the estimates of bias and the direction of the bias. It should be 
noted that the estimated bias for the Adolescent Immunization Status, Combination #| measure 
takes into account the findings of the medical record review, which indicate an overestimate. It is 


also possible that there was an underestimate of the actual rates related to excluding State Public 


Health Immunization Registry (MOHSAIC) data. 


Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures. 





Measure Estimate of Bias Direction of Estimate 

Adolescent |!mmunization Status, Combination #1 20.92% Overestimate 

Adolescent Well-Care Visits 18.75% Overestimate 

Use of Appropriate Medication for People with Asthma 0.00% None 
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Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 
measure. Table 2 shows that the Adolescent Immunization Status measure was not valid due to the 
validated rate falling below the 95% lower confidence limit reported by Community Care Plus. The 
Adolescent Well-Care Visits measure is not valid due to the use of 2002 service dates for the 


numerator. 


Table 2. Final Audit Rating for Performance Measures. 


Measure mi latelMcVereliam»t-halaye 
Adolescent |mmunization Status, Combination #1 Not Valid 


Adolescent Well-Care Visits Not Valid 

Use of Appropriate Medication for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by the MCO. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Calculation of the HEDIS 2004 Use of Appropriate Medications for People with Asthma 
measure was fully compliant with specifications. 


2. The data repository was very well-designed and had methodically arranged queries. Most of the 
queries were ‘make table’ queries which received the data online from the Amisys database. 
The data is then ‘frozen’ into the HEDIS repository built in MS Access. The source code was 
reviewed for accuracy and results were verified. 


3. The arrangement of the tables and queries was step-by-step and facilitated a detailed overview 
of the process. It also minimized the possibility of error. An example of such a step-by-step 
methodology was having a separate member span table, which was updated as soon as the MCO 
received changes in members’ status. This table calculated continuous enrollment in a detailed 
fashion and also ruled out gaps in enrollment. the EQRO conducted a validation test on the 
Asthma measure. Standard codes such as ICD-9 and UB-92 were checked for accuracy and 
updated versions. 


4. Up-to-date code versions were provided to Novasys by their auditor, HRA and were integrated 
well into the system for producing reliable results. 


5. There was excellent data integration with automated quality checks and reports. Data was 
collected from Amisys, the claims management system for membership, enrollment and 
encounter data. Pharmacy data was merged from ExpressScripts via Amisys into the warehouse. 
Controls for data checking and error handling were present within the Amisys system. 


6. The data resides in a secure location within Perot Systems, the owners of Amisys, in Dallas, 
Texas. The data is transferred through a secure ODBC connection. 


7. Information system policies are present within Novasys to maintain the integrity of the data. 
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Community Care Plus was responsible to feedback from the EQRO exit interview and report of 
preliminary findings by immediately recalculating and reporting the HEDIS 2004 Adolescent 
Well-Care Visits measure to the SPHA and EQRO; and by seeking assistance in improving the 
process of rate calculation and performance measure validation. 


AREAS FOR IMPROVEMENT 


Data from external databases such as the State Public Health Immunization Registry 
(MOHSAIC) for collection of adolescent immunization data was excluded from the rate 
calculation. The current rates are likely underestimated by 8% to 25% as a result of omitting 
MOHSAIC data. 


The rate for Use of Appropriate Medications for People with Asthma measure was significantly 
lower than the average for all MC+ MCOs. 


Documentation of the working mechanisms of the HEDIS repository to provide external 
observers and subsequent users with data field definitions and the workflow within the 
repository is needed. 


The data quality checks did not detect the incorrect dates of service for the Adolescent Well- 
Care Visits measure. 


Retention of copies of medical records used for performance measure development and 
validation is needed. 


RECOMMENDATIONS 


Community Care Plus should document the working mechanisms of the HEDIS repository and 
integrate them into the IS policies. 


Data from MOHSAIC for the Adolescent Immunization Status measure should be incorporated 
into measure calculation to produce rates in a reliable manner for comparison with other MC+ 
MCOs. This can be done by developing a data exchange procedure with the SPHA. Once a 
year, MCOs send a list of eligible members to the SPHA, which will return immunization data 
for the members. These procedures were outlined for the MCO in April, 2005. 


Maintain copies of medical records used for HEDIS hybrid rate calculation. This may be an area 
for oversight and including a provision where subcontracts for medical record reviews can 
specify a need to retain copies of medical records for audit and validation purposes. 


Continue the move to a structured data warehouse such as SQL Server from the current MS 
Access repository, as it will facilitate better data retrieval and analysis. 


Incorporate stronger review and validation processes into the rate production process and 
stronger MCO oversight for performance measure calculation. Training for staff of the MCO 
involved in the oversight and coordination of measure calculation is strongly recommended. 
The NCQA conducts several workshops on the process of measure calculation. 


Performance Management Solutions Group 179 
BHC a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Validation of Encounter Data 


FINDINGS 


The findings for the encounter data validation are organized according to the encounter data 


evaluation questions presented in the Technical Methods section for the encounter data validation in 


the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical Fields? 


For the Medical claim type, there were a total of 95,566 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


l. 
Z. 
3 


The Outpatient Claim Type field was |100.00% complete, accurate and valid. 
The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.99% valid. 
There were 5 invalid dates of service ranging from 01/05/2001 — 12/16/2003. 


The Outpatient Last Date of Service field was |00.00% complete and accurate, and 99.99% valid. 
There were 5 invalid dates of service ranging from 04/02/2004 — 04/24/2004. 


The Outpatient Units of Service field was 100.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100.00% complete and accurate, and 99.96% valid. 
Invalid procedure codes included 24 ““WW0025” and 14 “Z7000” codes. 


The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 55.52% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 34.09% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
23.02% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 
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For the Dental claim type, there were a total of 12,981 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 


accurate and valid. 


For the Home Health claim type, there were no encounter claims paid by the SMA for the period 


January |, 2004 through March 31, 2004. 


For the Inpatient claim type, there were a total of 2,/20 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 


The Admission Type field was 100.00% complete, accurate and valid. 


= @— 


The Admission Date field was 100.00% complete and accurate, and 93.90% valid. There were 
166 invalid dates ranging from 12/06/2003 — 12/31/2003. 


5. The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 97.79% (with 60 entries of 
“99999999"). Valid values were present 90.81% of the time. In addition to the invalid 
“99999999” entries, 190 invalid dates ranged from 04/01/2004 — 06/08/2004. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Se a 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (89.15%, 70.59%, 52.17%, 36.69%, 
respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 93.90% valid. There 
were 166 invalid dates of service ranging from 12/06/2003 — 12/31/2003. 


l1. The Last Date of Service field was 100.00% complete and accurate, and 91.84% valid. There 
were 222 invalid dates of service ranged from 04/01/2004 — 06/08/2004. 


12. The Revenue Code field was 100.00% complete, accurate and valid. 


13. The Units of Service field was 100.00% complete and 99.38% accurate and valid. There were 4, 
765 blank fields. 


For the Outpatient Hospital claim type, there were a total of 16,860 encounter claims paid by the 
SMA for the period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 


2. The Recipient ID field was 100.00% complete, accurate and valid. 
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The First Date of Service field was 100.00% complete and accurate, and valid. 
The Last Date of Service field was 100.00% complete and accurate, and valid. 


The Units of Service field was 100.00% complete, accurate and valid. 


SS oe ov 


The Outpatient Procedure Code field was 100.00% complete and accurate, and 61.82% valid. 
The invalid codes were 6,437 codes of ‘‘00000”’. 


7. The Revenue Code was |00.00% complete and 71.74% accurate and valid. There were 4,/65 
invalid codes of “00000”. 


8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 47.47% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was |7.6/% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
4.89% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was |.61% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were a total of 43,281 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, accurate and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Community Care Plus, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. There 
were very few errors encountered in the critical fields examined across all claim types. For the 
Medical claim type, the Outpatient Procedure Code field contained some invalid entries contained 
large proportions of invalid codes (see findings above). The Hospital Outpatient Procedure Code 
and the Revenue Code fields contained a large proportion of invalid entries. Although the 
Procedure Code is optional if the Revenue Code is between 300 — 319, the Revenue Code is always 
required|. For the Inpatient claim type, the Discharge Date field contained invalid entries, and there 


was no data in some of the Units of Service fields. 


* Personal communication with J udy Muck, Assistant Deputy Director, MC+ Managed Care, April 07, 2005. 
Performance Management Solutions Group 182 


a Division of Behavioral Health Concepts, Inc. 


2/16 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 
www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Community Care Plus 
demonstrated significantly lower rates than the average for all MC+ MCOs for the Medical, 
Inpatient, Outpatient Hospital, and Pharmacy claim types. This may be a function of provider panel 
composition or claims administration, but a lower rate of one claim type (e.g., Outpatient Hospital) 
would likely be associated with a higher rate of another claim type (e.g., Medical). The possibility of 
incomplete data cannot be ruled out given the consistent pattern of low rates across claim types. 


Another possible explanation is less access to care for members, or a healthier member population. 


The findings from the performance measure analysis also supports the possibility of missing data 
from the NSF/CMS 1500 claims data based on the significantly lower rates of administrative hits for 
the HEDIS 2004 Adolescent Immunization Status, Combination #| Adolescent Well-Care Visits, and 


Use of Appropriate Medications for People with Asthma measures. 


To What Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 
the period of January |, 2004 through March 31, 2004 for medical record review. Of the 95,566 
Medical encounter claim types in the SMA extract file for January |, 2004 through March 31, 2004, a 
total of 100 encounters were randomly selected. This was an oversample, as the minimum required 
sample size was 88 per MC+ MCO. Providers were requested to submit both medical records and 
claim forms or claim histories for review. There were 85 medical records (85.0%) and 39 claim 
forms (39.0%) submitted for review. Encounters for which no documentation was submitted were 
unable to be validated. The match rate for procedures was 59.0%, with a fault rate of 41.0%. The 


match rate for diagnoses was 68.0%, with a 32.0% fault rate. 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing or illegible 


information (n = 91). For the diagnosis description in the medical record, the reasons for diagnoses 
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not matching the SMA extract file were missing or illegible information (n = 44), or no match with 
the description of the symptoms based on the information in the medical record (n = 9). For the 
diagnosis code on the claim form, the reasons for diagnosis codes not matching the SMA extract file 
were missing or illegible information (n = 66), or incorrect information (n = 2). The reasons for the 
diagnosis descriptor on the claim form not matching the SMA extract file included missing or illegible 


information (n = 74), and an incorrect code (n = I). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = I 1), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 6), incorrect codes (n = 8), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = 4), and not enough information for coding (n = 6). For the procedure code on the claim 
form, the reason for procedure codes not matching the SMA extract file were missing or illegible 
information (n = 65), or incorrect codes (n = 4). For the procedure description on the claim form, 
the reason for procedure codes not matching the SMA extract file was missing or illegible 


information (n = 78). 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, Community Care Plus had difficulty submitting files in the requested file 
layouts for the performance measure validation and encounter data validation processes. Files were 
requested in flat file (comma-delimited), machine readable format. Microsoft Access files with no 
associated data were submitted in response to the request for performance measure extract files, 
and comma delimited files with special characters were provided in response to the request for file 


layouts in national standard formats. This precluded the validation of MC+ MCO encounter claims 
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data against the SMA encounter claims extract file. The following section summarizes the issues 


found when attempting to load files submitted for encounter data validation analysis. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


Steve Shelton, Vice President of Novasys Health forwarded the following files to BHC on December 
30, 2004 on behalf of Community Care Plus. A brief letter indicated that no unpaid claims data 
were available. This letter indicates “the pharmacy data is in the format used prior to October 
2004, but, as discussed earlier, we were unable to produce the medical and facility data in the 


format used prior to October 2004.” 


I. NS41230.txt 364KB Text Document [2/30/2004 4:26 PM 
Given that the NSF/CMS 1500 file layout includes Medical and Dental encounter claim types and that a sample of 
100 Medical encounter claim types was selected for retrieval, it is expected that there would be at least 100 claims 
represented in this file, based on the data from the SMA encounter claims extract file. All records submitted were 


able to be loaded for analysis. 


a: RX41230.TXT 6,562 KB Text Document 12/30/2004 10:04 AM 
All 18 records provided were able to be loaded for analysis, despite the presence of special and control characters 


in several fields. 


3. UB41230.txt 8815 KB Text Document | 2/30/2004 10:54 AM 


This file contained invalid characters and was not able to be loaded for analysis. 


STRENGTHS 


|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 


2. The critical fields evaluated for the Dental and Pharmacy claim types were 100.00% complete, 
accurate, and valid. 


3. Community Care Plus demonstrated significantly higher rates of validation of the SMA 
encounter claims extract file against the medical record for the diagnosis and procedure relative 
the average for all MC+ MCOs. This was likely related to the higher rate of submission of 
medical records by Community Care Plus providers for medical record review. Reasons for 
medical records not matching the SMA encounter claims extract file primarily involved missing 
or illegible information. 


4. Paid encounter data for Medical, Dental, and Pharmacy claim types were able to be loaded. 
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AREAS FOR IMPROVEMENT 


The findings of the present investigation and validation reflect more the limitations of the SMA 
and MC+ MCOs to submit data to the EQRO in machine-readable and non-corrupt format than 
the actual validity of the SMA encounter database. 


For the Medical claim type, the Outpatient Procedure Code field contained some invalid entries, 
resulting in a 99.96% rate of valid data in the field. 


For the Inpatient claim type, the Discharge date field contained invalid entries for the date field, 
resulting in a 90.81% rate of valid data in the field. The Units of Service field contained 99.38% 
valid data in the field. 


The Outpatient Hospital claim type had invalid codes in the Outpatient Procedure Code and 
Revenue Code fields. The Revenue Code field is always required. 


Community Care Plus demonstrated significantly lower rates than the average for all MC+ 
MCOs for the Medical, Inpatient, Outpatient Hospital, and Pharmacy claim types. 


Data for Inpatient and Outpatient Hospital claim types were not able to be loaded due to special 
and control characters not present in the standard file layout. Record type 6! appeared 
incomplete. 


RECOMMENDATIONS 


The MC+ MCO should examine the rate of claims per |,000 members across claim types and 
the rate of rejected claims for each claim submission format (UB-92, NSF/CMS 1500, NCPDP 
3.0) over time to examine the consistency in claims submission and identify issues for data 
submission. The access to care should also be examined as a possible reason for the lower 
rates of encounter claims per |,000 members. 


Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 


Examine and revise as needed internal system edits for invalid procedure codes in the NSF/CMS 
1500 file layout and run validity checks after the programming of new edits. 


For the Inpatient claim type (UB-92 file layout), improve the rate of valid Discharge Dates to flag 
invalid entries of “99999999”, and ensure completion of Units of Service field. 


For audit purposes, submit extract files for performance measures and encounter data in the 
requested file layouts with the requested documentation of files. 


MCO Compliance with Managed Care Regulations 


METHODS 


Objectives, technical methods, and procedures were described previously in this report. This 


section describes the documents, data, and persons interviewed for the Monitoring Medicaid 


Managed Care Organizations protocol for Community Care Plus (Community Care Plus). The 


EQRO reviewed documentation between December |, 2004 and February 28, 2005. On-site 
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review time was used to conduct follow-up questions, review additional documentation made 
available by Community Care Plus, and provide feedback and recommendations regarding 


compliance with federal Medicaid Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MC+ MCO, 
Community Care Plus was requested to provide the following documents on-site: 


e Member Handbook 
e Provider Handbook 
e Provider Contracts 
e Grievance and Appeal Policies 
e Samples of both Grievances and Appeals related to member and providers 
e Fraud and Abuse Policy 
Additional documentation made available by Community Care Plus: 


e Medical Management — Low Birth Weight Babies Report 

e Member Newsletter — Spring 2005 

e Provider Monthly Payment Schedule FQHC/RHC Services 

e 2004 Member Services Monthly Metrics 

e Community Care Plus Instructions for Submitting Policies and Procedures — 2005 
e Required Policies and Procedures Review Order — 01/2005 

e = Policy Tracking Log 


Unity Managed Mental Health supplied the following documents regarding their services to 
Community Care Plus members: 


e Over/Under Utilization Report 

e Use of Services Trends 

e Readmissions Report 

e 2004 Denial/Appeal Report 

e 2004 Grievances Report/Resolution Timeliness 
e 2004 Adverse Events Outcomes Report 

e Pregnant Women Care Coordination Report 

e 2004 Provider Record Survey Results 

e Depression Treatment Standards 

e 2004 Provider and Member Satisfaction Reports 
e UMMH ADD Best Practices Screening/Treatment Protocol and Medication Guideline 
e Member Transition and Care Policy 
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Interviews 


The following individuals were interviewed on March I, 2005 during the on-site review: 


Plan Administration Beverly Thompson, Director, Medical Management 
Jerry Linder, President and CEO 

Cris Cristea. Chief Operations Officer Provider Services 

Dr. Parikh, Chief Medical Officer Cris Cristea, Chief Operations Officer 

Lee Kneibert, QI Manager Kathy Mocca, Manager, Network Development 


Quality Assurance 


Cris Cristea, Chief Operations Officer Mental Health 
Lee Kneibert, QI Manager Scott Frederick, PhD, Director, Managed Mental 
Health 


Marge Viehland, RN, Manager, Utilization 
Management and Quality |mprovement 
Kathy Mocca, Manager, Network Development 


Member Services/ Case 

Management/ Utilization Management 

Vicki Eisenbeis, Director, Claims/Member Services 
FINDINGS 

Enrollee Rights and Protections 
Community Care Plus Member and Provider Services staff exhibited an understanding of the 
requirements of the regulations concerning members’ rights and responsibilities. They actively 
attempted to engage all new members, to distribute member information material, and to 
participate in community-sponsored events that were of interest to members. These staff were 
aware of and encouraged the use of interpreter services and TTD lines for members who did not 
speak English or had a physical disability. The Member Services staff were enthusiastic about their 


role in assisting their members in receiving the healthcare services they needed. 


Ratings for compliance (15.4%) with regulations concerning Enrollee Rights and Protections were 
the result of a failure to develop and submit MC+ Managed Care Program required policy and 
procedures. When questioned about a number of policies relating to member rights, the MCO 
responded that these policies were “under development.” One policy shared was on the topic of 
Fraud and Abuse. It appeared to meet all of the requirements of the regulation and MC+ Medicaid 
Managed Care contract. The policy was well written and appeared to be thorough. This policy had 
not been sent through the MCO internal review process and had never been submitted to the SMA 
for their review and approval. The MCO did share their “Policy Tracking Log” to validate that an 
organizational method existed for informing staff about the need to develop, review, and submit 
policy. It did not appear that the MCO utilized this tool effectively as no dates were recorded in 
this document. Another area of concern was that this MCO had changed locations in November 
2004. At the time of the on-site review all MCO specific documents, including brochures that were 


given to members and prospective members, and distributed at sites such as county Family Support 
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and Children’s Division offices, included the previous address and telephone number. MCO staff 


could not verify if updated materials had been printed or distributed. 


Quality Assessment and Performance Improvement 

Access Standards 

Community Care Plus had an adequate provider network. The Provider Services staff maintained 
strong working relationships with providers to assist with problem solving. The MCO believed that 
these relationship building efforts assisted in preserving their network. The Provider and Member 
Services sections of the MCO worked together to ensure that members received timely 
appointments, PCPs that best met their needs, and access to specialists. They assisted members and 


worked directly with providers so that the best healthcare was available. 


Ratings for compliance with Access Standards (23.5%) reflected responses from staff that were 
vague and policy that was not complete. Staff and policy were not clear about the use of specialists 
as a PCP in the situations specified in the regulations. Other problems regarding access to care 
revolved around undeveloped, incomplete, and unapproved policy. No approved policy was in place 
regarding prior authorizations, which was an essential component of managed care. The policy had 
been submitted to the SMA, but was returned for revisions in July 2004. No further action to 
obtain policy approval was observed. Policy surrounding members obtaining emergency and post- 
stabilization services did not meet requirements. It appeared the MCO required prior authorization 
for some types of services, even in an emergency, if a member used an out-of-network facility. One 
of the ten (10) counties in this MC+ Region was not included in information regarding obtaining 
emergency services and there were no in-network emergency service facilities listed in the Member 
Handbook. This county had two hospitals, but neither was included in the list of facilities where 
members could obtain emergency room services. Undeveloped and conflicting policy and member 


information could create confusion for members and providers. 


Structure and Operation Standards 

Ratings for compliance with Structure and Operation Standards regulations (40%) reflect the need 
for policy and procedures. Practice that could be determined through the interview process 
appeared to be in place. All outstanding issues concerned submission of completed policy. With 


the submission of required policies this section of regulations will be in compliance. 
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Measurement and Improvement 

An area of marked improvement for the MCO was the adoption, dissemination and application of 
clinical practice guidelines. The MCO recognized guidelines established by professional 
organizations pertaining to prenatal care and asthma. Community Care Plus has joined Medical 
Directors from other MC+ MCOs in the adoption of these standards. The MCO encouraged the 
use of clinical guidelines by providers. During interviews the Provider Relations staff denied 


knowledge of this practice. They would benefit from training in this area. 


The Medical Director discussed his proactive involvement in peer-to-peer contacts when there was 
a problem or question regarding medical necessity or denial of requested authorizations. The MCO 
developed a system of applying InterQual standards in utilization review, and applied nationally 
accepted clinical guidelines in medical decision-making. The MCO had not developed or 


implemented an actual definition of medical necessity. This policy was required. 


Ratings for compliance with Measurement and Improvement (18.2%) reflect a number of problems 
including lack of required policy, an inability to provide adequate data, and a lack of understanding by 
staff of requests for technical information. Community Care Plus exhibited a number of problems in 
the area of Measurement and Improvement throughout the EQR. Understanding the parameters of 
requested information and data was a pervasive problem. This lack of understanding and lack of 
internal organizational communication led to problems in providing required information for 
validating performance measures and validating encounter data. In several cases the EQRO 
attempted to provide technical assistance to ensure proper data submission, but this had little 
positive impact on the outcome. Community Care Plus did not have complete policy regarding 
these regulations, which created additional barriers to compliance with their MC+ Medicaid 


Managed Care contract and the federal regulations. 


Grievance Systems 

Community Care Plus was operating a grievance and appeal system as required. Both grievance and 
appeal files were reviewed. The MCO appeared to initiate and complete this process within 
required timeframes. The decision-making observed in the files reviewed indicated that grievances 
and appeals were researched thoroughly, and decisions were made with regard to the member’s 


well-being as a primary concern. 
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The ratings for Grievance Systems regulations (50%) indicated lack of complete or approved MCO 
policy. The areas of deficiency included lack of approved notifications for members, and the 
admission by Community Care Plus staff that this policy was still “under development.” Policy 
reviewed on-site continued to refer to the “Complaint, Grievance, and Appeal” process. The MCO 


staff admitted that this policy was out-of-date and that new policy was not completed. 


Policy and Procedures 
Written policy and procedures were a serious problem area for the MCO. Numerous examples of 


contradictory or inaccurate information were found. 


|. In the Member Handbook, approved by the SMA in January 2005, there is a statement “The 
following page contains a list of hospitals where you can get ER services.” There is no 
clarification with this statement about going to the nearest hospital or emergency room. The 
listing did not contain one hospital for the geographically largest county in the region. The 
county was not included in the listing. One hospital that was listed in St. Louis County was 
Ranken Jordan Pediatric Rehabilitation Center. This facility had recently received hospital 
accreditation, but did not have Emergency Room services. The facility only provided 
rehabilitation, transition, and care services for seriously involved pediatric patients. The address 
and phone number listed for Ranken Jordan was incorrect as the facility moved in the fall of 
2004. 


2. On page 3 of the Policy and Procedure Manual there was a list of non-precertification services 
and procedures. The list did not include all exemptions required by the MC+ Medicaid Managed 
Care Contract (e.g. emergency services). A sentence following the list states, 
‘...precertification is required for anything not listed on the non-precertification list.” 


3. The signature page for the Policy and Procedure Manual was dated 2002. Individual policies, if 
signed and dated, showed signatures and dates from 2000 and 2001. 


4. There was no policy covering the comprehensive benefit package. 


5. The Prior Authorization Policy was dated March 2002. This policy had been submitted to the 
SMA and was returned for revisions. No further action was observed. 


6. It was noted that the Denials and Appeals Policy was to be resubmitted to the SMA for 
approval, but this had not occurred. 


7. Policies regarding transfer and transition of care were outdated. These were to be resubmitted 
to the SMA for approval, but this had not occurred. 


8. The Subcontractor Oversight Policy was dated September 2002. 


9. Fraud and Abuse policy was shared. It was clearly written, thorough, and concise. It appeared 
to include all required language. The policy had not been approved internally, nor was it 
submitted to the SMA for final approval. 
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Summary and Follow-up 

Behavioral Health 

Community Care Plus made a transition in behavioral health contractors from Magellan to Unity 
Mental Managed Health (UMMH) in January I, 2004. A smooth transition was effected through use 
of the MCO NovaSys information system. High risk members and hospital discharges were 
identified and followed. Claims data was utilized to identify Magellan providers. Behavioral health 
services were continued using out-of-network providers until new contracts were negotiated or a 
therapeutic transition to in-network providers could be achieved. Ninety-five percent of the 
Magellan providers were retained in the UMMH network. The contract with UMMH resolved a 


number of issues in the area of behavioral health services. 


A follow-up issue involved improving the number of mental health appointments members attended. 
The BHO expressed the belief that keeping appointments was related to the relationship developed 
with the provider. Therefore, UMMH utilized Medicaid dedicated providers as often as possible. 
Case managers contacted members to remind them of appointments. In-home services were 
scheduled with members who chronically missed appointments due to transportation problems or 
an unwillingness to participate in therapy. Home visits appeared to have been instrumental in 
reducing the number of hospital admissions in 2004. In the first quarter of 2004 there was a 
noteworthy upward spike in in-patient readmissions. In-home treatment and additional case 


manager follow-up brought this number down throughout the rest of the calendar year. 


UMMH added nurse practitioners as physician extenders to the provider network in an effort to 
address the shortage of child psychiatrists. Use of traveling psychiatrists and in-home services also 


supplemented the UMMH network in rural areas. 


UMMH developed a “Clinical Guide for Major Depression” based on national clinical guidelines. The 
BHO included variables in treatment based on the population served and locally accepted practices. 
UMMH also developed clinical guidelines for ADHD based on a combination of local variables and 
national guidelines entitled UMMH Attention Deficit Disorders — Best Practice Screening and 
Treatment Protocol. This protocol was based on American Academy of Pediatrics: ADHD Practice 
Guidelines, and Children and Adults with ADHD (CHADD) guidelines. The protocol included 
ADHD Medication Guidelines. 
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UMMH initiated a pregnant women study focusing on prenatal, pre-delivery, and post partum 
depression. Eight-two percent of the women in the study had been identified by Community Care 
Plus case managers as members who needed mental health services. There was good 
communication between UMMH and Community Care Plus case managers for coordination of care 


including outreach and referral. 


UMMH used the data they collected to inform their system on trends, utilization rates, survey 
results, and service outcomes. They used the reports produced to provide analysis and to create 


quality improvement actions. 


Low Birth-Weight Babies Report 

Community Care Plus developed a report in an attempt to utilize available data regarding the 
incidence and cause of low birth-weights. The report developed collected data to provide 
information on the number of low birth-weight infants born during 2004. The story these numbers 
told was difficult to discern. It was unclear what questions the data attempted to answer or how 
the gathered data was analyzed. How this information was evaluated to make recommendations for 


program development or enhancement remained vague. 


EPSDT and Lead Poisoning Education 

A number of recommendations or observations regarding areas for improvement were identified for 
the EPSDT program. Community Care Plus provider relations staff reported that they continued to 
provide education to PCPs, particularly general practitioners, family practitioners, and pediatricians, 
on the importance of completing EPSDT examinations and reporting on the outcomes. Community 
Care Plus completed an audit and was able to report significant improvement in reporting. They 
expressed a continued commitment to improving the completion and accurate reporting of EPSDT 


examinations. 


Community Care Plus’s Utilization Management department continued to track reported elevated 
lead levels. Case managers tracked individual cases and monitored ongoing progress. It was unclear 


if the MCO intended to develop a performance improvement project based on this data. 
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New Member Contacts 

Community Care Plus staff continued to make attempted contacts by telephone, as well as mailing 
packets to new members. The MCO used any call or contact from members to ensure that they 
had correct contact information in their system. If these calls were first contacts they used them to 
educate members on plan benefits. It was not clear if the MCO had developed a “shadow system” 
for retaining separate member contact information. The Member Services staff considered the issue 


of maintaining accurate member contact information a problem. 


STRENGTHS 


|. The behavioral health organization (BHO) serving Community Care Plus members was Unity 
Managed Mental Health (UMMH). These two organizations developed a strong partnership 
working specifically to the needs of the MCO’s members. UMMH defined best practices and 
clinical guidelines for two issues with which Community Care Plus members struggle. These 
guidelines addressed depression management and ADHD. UMMH made a practice of approving 
and using in-home therapy to meet the needs of the member population. They used this service 
in rural areas when members needed transportation. They also authorized in-home treatment 
when members (adolescents for example) did not adhere to treatment plans by attending office- 
based sessions. Their focus was on providing service to the member. 


2. The provider relations section of Community Care Plus focused on being available to the 
provider to answer questions, to assist in problem solving, and to ensure that providers served 
members according to state and federal requirements. Provider Relations field staff that visited 
offices frequently. The field staff also did network development to ensure that access standards 
were met. They reviewed appointment calendars and performed other spot checks to learn if 
provider offices were meeting their contractual obligations. The Provider Services department 
exhibited an understanding and commitment to ensuring that members’ rights were met and 
protected. 


3. Throughout interviews with Community Care Plus staff, it was obvious that each section was 
aware of their integral part in providing effective and efficient services to MCO members. Even 
though specific sections of the organization had their own assignments, they were all aware of 
their role in providing member services as their ultimate goal. Individuals within the 
organization exhibited an understanding of MCO’s goal of providing good member services. 
The staff was aware of the influence their actions had on the organization and its members. 


4. Community Care Plus staff understood the importance of continually assessing their members’ 
needs and using this information to improve their product. 


AREAS FOR IMPROVEMENT 


|. Work with DHSS to obtain data downloads from MOHSAIC to enable the MCO to obtain data 
on its member immunizations. A contact name was provided to Community Care Plus during 
the site visit in 2004 and was requested again in 2005. MCO follow-up was needed. 
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lt appeared that Community Care Plus did not always effectively use its resources to complete 
documentation of their practice and submit necessary policy to the SMA within required 
timeframes. Many of the positive practices that occurred at Community Care Plus were lost, as 
these practices were not reflected in current policy or procedure. 


Community Care Plus did not have medical necessity defined in their policy. Community Care 
Plus staff was able to discuss how they define medical necessity in practice, and how they use 
InterQual criteria and standards. However, they did not have these practices documented in 
policy as required. 


The grievance and appeal process as reflected in the Member Handbook complied with MC+ 
Managed Care standards. However, the policy seen on-site continued to reflect the 
“Complaint, Grievance, and Appeal” process and was not up-to-date. Community Care Plus 
staff shared that continued enhancement of this written policy and practice was occurring, but 
documentation was not complete. 


Written materials shared for the on-site review contained Community Care Plus’s prior address 
and telephone number. It was not clear if current documents available to the public, such as 
those placed in local Family Support or Children’s Division offices had been updated. Brochures 
and other information encouraging members or potential members to contact Community Care 
Plus should contain correct contact information. 


RECOMMENDATIONS 


Raise the importance of complying with documentation requirements to the same standards as 
those reflected in the daily practice within the MCO. The MCO must give attention to the 
details of developing clear and complete policy and procedures. On-site interviews provided 
evidence that staff had substantive working knowledge of the MC+ Managed Care Program 
requirements and the federal regulations regarding providing services to members. A systematic 
method for review and approval of policy should be developed. The MCO must demonstrate 
that it can put its knowledge of both MC+ Managed Care Program and federal requirements 
into writing to ensure consistent compliance with regulations. 


Continued MCO development in the area of utilizing available data and member information to 
drive change and create opportunities for organizational growth and development. 


Continued partnership with UMMH to ensure that communication between behavioral health 
and PCPs is strengthened. Look at UMMH program improvement projects, and trend 
development reports. Utilize available data as a model to replicate these efforts throughout the 
MCO. 


Complete the development and implementation of written policy regarding medical necessity. 
Continue to develop a process for monitoring provider adherence to clinical guidelines and the 
introduction of findings into a system for provider profiling. Provide training and information 
involving Provider Relations staff. Include this staff in the dissemination of clinical guidelines to 
providers. 


Accurately articulate the questions to be answered, or the study problems at the beginning of a 
project. State this information in written reports. Develop and state a hypothesis about the 
issue or problem being studied. Create narrative information to make the data and 
accompanying charts and graphs understandable. 
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6. Awritten plan of action is needed to create a methodology for attacking the problem of EPSDT 
performance and identifying members with elevated lead levels. Include current screening rates, 
educational input, and tracking on screenings reported. An analysis about the origin of the 
problem could include questions that clarify if screenings occur, and if so, are they reported. 
This information would compliment quality improvement activities for both elevated lead levels 
and EPSDT. A plan of action should also contain a system for validation of EPSDT claims 
submitted by providers. Use and track information gathered by contacts or attempted contacts, 
by telephone and mail, to provide data that can inform the system, and assist in quality 
improvement efforts. Information gathered should be retained in a “shadow system” to ensure 


its capture. 
Performance Management Solutions Group 196 
8. HC a Division of Behavioral Health Concepts, Inc. 
2716 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MISSOURI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 





MERCY HEALTH PLAN 


The previous sections of the 2004 EQRO report present the purpose and objectives technical 
methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 


Document Review 

Mercy Health Plan supplied the following documentation for review prior to the site visit: 

NCQA Quality Improvement Activity Form: Improve the Health Outcomes of Babies Born to 
Missouri Medicaid High Risk Pregnant Mothers 

NCQA Quality Improvement Activity Form: Improving the Quality-of-life of Members Living with 


Asthma 


Additional documentation for each of the two projects validated was reviewed on-site, including but 
not limited to: 


Charter 

Policies and procedures 
Methods 

Care plan 

Assessment tools 
Surveys 

Training tools 
Interviews 


Project Leaders, participants, and the Medical Director were interviewed on-site by the EQRO 
Project Director on Wednesday, March 2, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided as were references for logic 
model development and health services research methods. The following questions were addressed 


for specific performance improvement projects on-site: 
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Improving Quality of Life of Members Living with Asthma 
What is MCD? 


What is the analysis plan? 

lt appears as though the intervention is to identify and stratify. How well was this implemented? 

How is quality of life and improved health measured through encounter/claims data? 

e  |t appears that quality of life is being defined as inpatient admissions and ER visits. How is improved health 
being measured? 

e Health visits were mentioned as a rationale for conducting the study. Was this measured? 

e How is the Use of Appropriate Medications helping to identify the effectiveness of the identification and 
stratification process? 

e How were members identified? Stratified? Based on what criteria? 


High-Risk Pregnancy 
e What is the purpose of following the infant hospital length of stay, and the APGAR scores? 


e What is being done with these data? 

e What was the difference between “identified” and “risk stratified’”? 

e How effective was the identification and stratification? 

e What was the re-measurement rate for 2004 on the number identified? VVhat were the barriers to 
identification? 

e Why was Tier | discontinued? 

e The High Risk Pregnancy program policy and procedure for stratification of pregnant Medicaid members 
into level of risk was referenced, but not included. 


FINDINGS 

The first PIP evaluated was “Improving Quality of life of Members Living with Asthma”. As stated in 
the narrative, “the project was designed to impact the health and quality of life of Mercy Health Plan 
members living with asthma.” There was an excellent review of the literature to develop the 
rationale for the topic and select the most appropriate interventions. The goal of the project was to 
improve quality of life and health care outcomes. The intervention involved the identification and 
stratification of members to various levels of intensity of care based on the intensity of their needs. 
All MC+ Members had equal chances of being included in the study and intervention for 
identification, needs assessment, and assignment to various disease management interventions. 
Although there was no study question in the quality improvement activity form used for 
documenting this PIP, the study question was clarified on the site visit documentation and interviews 
to be, “Does the identification and stratification of MC+ members with asthma into various levels of 


intervention lead to improved health outcomes and quality of life?” 


Indicators included the HEDIS Use of Appropriate Medications for People with Asthma measure, 
the number of inpatient admissions per |,000 members, and the number of emergency department 
visits per 1,000 members. There was a good presentation of methods of calculating measures and 
the benchmarks used for comparison. There were references to HEDIS measures, but the 


denominator definitions were not consistent with HEDIS technical specifications. The continuous 
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eligibility criteria were not used so as not to exclude members from the intervention, which is 
acceptable and preferable, as it does not exclude members who may benefit from an intervention 
based solely on measurement criteria. Eligibility criteria were very well specified and defined in 


additional documentation provided on-site. 


The role of the routine healthcare visits measure mentioned in the rationale for the study was 
unclear, but may be used as a measure of accessibility. There were no long-term outcomes 
indicated, but the connection between the measures of process and outcomes was able to be 
determined. It is possible that ER and inpatient utilization were considered measures of quality of 
life. Additional functional measures such as activity limitation for those in various disease 


management groups would be excellent additional measures for this study. 


Demographic information of participants was not described and it was unclear whether MC+ 
Members only were part of the study. From the study rationale and the data table, it was 
ascertained that MC+ members were participating in the study. However, demographic description 
of the study population should not be assumed and should be fully described in the narrative. 
Additional documentation provided on-site indicated that the MC+ members and commercial 
populations were examined separately, so that the findings of the study pertained only to MC+ 
members. The data collection approach appeared to capture all enrollees to whom the study 
question applied. Participants were identified through emergency department, hospital, PCP 


utilization, and pharmacy claims data. The study included all members meeting the study criteria. 


Stratification of analyses by level of need (none, low, moderate, high) could be useful in comparing 
groups on outcomes at baseline and over time. Although it was possible to determine the source of 
some of the data, this should be more clearly stated in the Quality Improvement Activity (QIA) 
Form. Data collection procedures were not specifically described and should be specified. Some of 
the measures (HEDIS) were calculated using methods with established reliability and validity. 
However, the available information did not provide reliability validity information for some of the 
measures. There was no data analysis plan identified in the narrative. Planned data analyses should 
attempt to identify the significance of changes since baseline. Since the project was underway, there 


were no analyses of findings. 
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This study has moderate potential for producing credible findings. Some minor modifications in data 
analysis would result in highly credible findings that could be used on a continuous basis for 
improvement in functional outcomes and quality of life. The QIA Form provided limited information 
about the entirety of the quality improvement and PIP that was implemented. However, it was clear 
upon interviews and site visit documentation that this is an active and viable PIP with potential for 


identifying real improvement and evaluating the success of interventions over time. 


The second PIP evaluated was, “High Risk Pregnancy.” “The purpose of the study is to evaluate 
changes in operations that would lead to increased identification of pregnant Medicaid members for 
risk stratification and placement in the High Risk Pregnancy Program.” The rationale was that the 
timely identification of pregnant members at risk can prevent negative health outcomes and improve 
birth outcomes (e.g., estimated gestational age, birthweight, newborn length of stay, and APGAR 
scores) for children. Low birthweight (LBW) and gestational age were linked to infant mortality in 
the background discussion. There was no study question stated, but it could have been, "Does the 
identification and stratification of pregnant MC+ members into a 3-tiered model of prenatal care 


case management improve the pre- and post-natal outcomes of MC+ members?" 


No information about the participants included in study was presented in the narrative. From the 
topic of this project it can be inferred that only women participated in the study. Details need to be 
included in the narrative regarding demographics of study population. It appeared as though all 
MC+ members were targeted for the intervention and had equal chances of being included in the 
intervention and study. Sampling was not used. All pregnant members had equal chances of being 
screened and selected for the intervention. It is not entirely clear how study participants were 
being identified or recruited. This is important, as one of the key indicators of the implementation 


of the intervention is the number that was identified for the prenatal care program. 


A very thorough description of measures and methods of calculation was presented in the narrative. 
Measures and methods of their calculations were very well summarized and explained. They were 
relevant to the stated purpose, although the APGAR scores are not likely to be impacted by the 
intervention. There were several sources of data identified on the Quality Improvement Activity 
form, but there was no association of the source of the data with the measure. It is necessary to 
link the source with each measure if there is more than one source of data. It appears that Mercy 


Health Plan has done re-measurement of indicators specified in the study, and they plan to continue 
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this project in the future. Given that the intervention (identification and stratification) was applied 
to the entire population of eligible (pregnant) members, the entire population should be included in 


the outcome measures. 


Risk stratifications were well defined and incorporated medical judgment. There was a risk 
assessment form shown, but no instructions for use or implementation. The use of a standard form 
is important for standardizing data collection and should be continued. No data analysis plan was 


identified in the narrative. 


It is unclear how the barriers were identified, however Mercy Health Plan staff seemed to develop a 
reasonable interventions to address the barriers encountered. There should be an explanation for 
the reason for discontinuing the Tier | intervention rather than a list of barriers alone. The main 
intervention stated in the purpose and the hypothesis was identification and stratification. The study 
narrative will need to address and consider the low rate of identification of members (36% relative 
to the stated goal of 85%). If the intervention is not being effectively implemented, there will not be 
strong enough findings to conclude that it is effective even if it truly is effective. On-site discussion 
revealed that there were issues with implementation that were appropriately being addressed for 


the next cycle. 


This project has moderate potential for producing credible findings with minor modifications in the 
outcome measures and improvement in the implementation of the intervention (identification and 
stratification). There was a very good presentation of study rationale and description of measures 
calculation. When conducting the re-measurement, it is recommended that Mercy Health Plan 
calculate the outcome measures (three through seven) using the entire population of MC+ eligible 
members, recalculate the baseline using the same definition, and conduct statistical analyses for 
comparison. There may not yet be statistically significant improvement, but continue to refine the 
intervention to improve the rate of reaching members (36%) each year. Explain the data analysis 
plan and limitations of the findings given the success of implementing the intervention, and provide a 


rationale for the discontinuation for the Tier lintervention and any implications on the findings. 
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STRENGTHS 


Performance improvement projects were designed as long-term, continuous programs of quality 
improvement designed to identify large groups of members early, identify their needs, determine 
barriers to access, and test the effectiveness of specific disease management programs. Mercy 
Health Plan’s implementation of performance improvement projects exemplifies continuous 
quality improvement programming. The PIPS underway during 2004 have the potential for 
identifying Best Practice for MC+ Member Care. Findings of effectiveness of the PIPs will likely 
produce Best Practices for widespread application for MC+ Members. 


Study topics were well defined. Literature reviews were well summarized to define the problem 
and study topic, and were used to identify appropriate interventions to be implemented by the 
MCO. 


The study rationales were supported by literature reviews and baseline data. 


Interventions were appropriate and were likely to account for change in most of the measured 
outcomes. 


Relevant measures were identified for change. 


Specific criteria were used for identification and stratification of members into disease 
management programs. These criteria were developed in collaboration with the Medical 
Director and staff, using clear operational definitions for interventions at various levels of need 
as well as allowing for clinical judgment and practitioner input into levels of risk assignment. 


Medical expertise was incorporated from the Medical Director and provider network 
practitioners. 


AREAS FOR IMPROVEMENT 


Re-measurement and presentation of findings could be improved. 


2. Documentation of measures of the implementation (process measures) is important. There was 
good discussion of the issues with implementation and Mercy Health Plan staff understand the 
need to include implementation measure as well as improve implementation before evaluating 
the effectiveness of the intervention. 

RECOMMENDATIONS 

|. For the asthma PIP, analyze data on the entire population rather than high risk members only. 

2. Compare pre- and post-intervention data using the entire population eligible for identification, 
regardless of the risk group. Stratification of analyses can then be conducted for pre- post- 
intervention analyses within groups (no, low, moderate, and high risk). 

3. Continue using state data as comparison, as this is the most proximal (most comparable to the 
same population being evaluated) level of data available. 

4. Continue to document the success of the implementation of the intervention through the 
incorporation of functional measures and member feedback. 
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5. Include tests of statistical significance to compare pre- and post-intervention changes. Chi- 
Square analyses and upper and lower confidence intervals could readily be calculated using the 
available data. 


6. When submitting PIPs for EQRO, submit the following sections: Charter, policies and 
procedures, methods, care plan, assessment tools, surveys, QIA form, and training tools. 


Validation Of Performance Measures 


METHODS 


This section describes the documents, data, and persons interviewed for the Validation of 


Performance Measures for Mercy Health Plan. Mercy Health Plan submitted the requested 


documents on November 23rd, 2004. The EQRO reviewed documentation between November 


23rd, 2004 and February 28th, 2005. On-site review time was used to conduct follow-up questions 


and provide feedback and recommendations regarding the performance measure rate calculation. 


Document Review 


The following are the documents reviewed by the EQRO: 


The Information Systems Capability Assessment (ISCA) submitted by Mercy Health Plan 

The Baseline Assessment Tool (BAT) submitted by Mercy Health Plan 

Healthcare Research Associates’ (HRA) HEDIS Compliance Audit Report for 2004 

Mercy Health Plans Information Systems (IS) policies and procedures pertaining to HEDIS 2004 rate 
calculation 

Mercy Health Plans IS policies on disaster recovery 

CRMS & HEDIS project plan 

NCQA HEDIS software certification report 

Medical record review process flowchart 

CareEnhance Resource Management System (CRMS) data warehouse and Health Plan Reporter (HPR) 
Manual 


The following are the data files submitted by Mercy Health Plan for review by the EQRO: 


Adollmms Denominator.txt 

Final, Adolescent_Imm_Denomiator_File.txt 
Final, Adolescent_Imm_Numerator_File.txt 
Final_ Asthma_Denomiator_File.txt 

Final, Asthma_Numerator_File.txt 

Final_ Well_ Care Denomiator_File.txt 
Final_ Well. Care _Numerator_File.txt 

MC File Specification.doc 
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Interviews 

The EQRO conducted a site visit and interviews at Mercy Health Plan with Patricia Snodgrass, 
Charles McLaughlin, Corey Waters and Dave Reisinger on Wednesday, March 2nd, 2005. This 
group was responsible for calculating the HEDIS 2004 performance measures. The objective of the 
visit was to verify the methods and processes behind the calculation of the three HEDIS 
performance measures. The information systems (IS) management policies and procedures for rate 
calculation were evaluated consistent with the CMS Final Protocol for the Validating Performance 
Measures. This included both manual and automatic processes of information collection, storing, 


analyzing and reporting. 


FINDINGS 

Mercy Health Plan calculated the Adolescent Immunization Status, Combination #I and the 
Adolescent Well-Care Visits measures using the Hybrid Method. MCO to MCO comparisons of 
the rates of Adolescent Immunization Status Combination #1, Adolescent Well-Care Visits, and Use 
of Appropriate Medications for People with Asthma measures were conducted using two-tailed z- 
tests. For comparisons that were statistically significant at the 95% confidence interval (Cl), the z- 
score (z), the upper and lower confidence intervals (Cl), and the significance levels (p < .05) are 


reported. 


The rate for the HEDIS 2004 Adolescent Immunization Status, Combination #| reported to the 
SMA and the State Public Health Agency (SPHA) by Mercy Health Plan was 19.22%. This was 
comparable to the statewide rate for all MC+ MCOs (14.36%; z = -.37; 95% Cl: 4.36%, 34.08%; n.s.). 


The rate for Mercy Health Plan for the HEDIS 2004 Adolescent Well-Care Visits measure was 
24.82%, which was significantly lower than the statewide rate for all MC+ MCOs (30.13%; z = -.62, 
95% Cl: 19.94%, 29.70%; p < .0O1). The 2004 HEDIS rate for Mercy Health Plan for the Use of 
Appropriate Medications for People with Asthma was 60.63%, which was comparable to the 


statewide rate for MC+ MCOs (63.92%, z = -.30; 95% Cl: 56.99%, 64.27%; n.s.). 
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The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

For all three measures, Mercy Health Plan was found to meet all criteria for producing complete and 
accurate data (see Attachment V: Data Integration and Control Findings). There were no biases or 
errors found in the manner in which Mercy Health Plan transferred data into the repository used 


for calculating the HEDIS 2004 measures. 


Documentation of Data and Processes 

Mercy Health Plan used NCQA-certified software from McKesson Inc, for the sampling and 
calculation of HEDIS measures. The EQRO was provided with a demonstration of the Health Plan 
Reporter (HPR), the application module for rate calculation, along with the CareEnhance Resource 
Management System (CRMS) data warehouse. Data and processes used for the calculation of 
measures were adequate (see Attachment VII: Data and Processes Used to Calculate and Report 
Performance Measures). Mercy Health Plan met all criteria that applied for all three measures. The 
two criteria that were not met involved the use of statistical significance testing to document 
changes in performance over time and using this information to support claims of improvement or 
stability in performance over time. Although many auditors conduct statistical tests to examine 
various aspects of data validity, such tests should be used by the Mercy Health Plan to assess the 
significance of change related to quality improvement activities and operational changes. The use of 


NCQA-certified calculation software is considered adequate for calculating the rates. 


Processes Used to Produce Denominators 
Mercy Health Plan met all criteria for the processes employed to produce the denominators of all 
three performance measures (see Attachment X: Denominator Validation Findings). This involves 


the selection of members eligible for the services being measured. The denominator file for the 
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Adolescent Immunization Status measure contained a total of 411 members. These members 
represented the sample drawn from the eligible population. No files were provided containing the 
eligible population. There were no members represented more than once. The dates of birth and 


enrollment were valid. 


For the Adolescent Well-Care Visits measure, the denominator file contained a total of 411 and 


there were no duplicate entries. The dates of birth and enrollment were within the valid ranges. 


For the Use of Appropriate Medications for People with Asthma measure, there were a total of 447 
eligible members. There were no duplicate members included. The dates of birth and enrollment 
were in the valid ranges. There were a total of |,836 events that were mapped by ICD-9, CPT, 
GCN and Revenue Codes. 


Processes Used to Produce Numerators 

Data ranges for the qualifying events (e.g., immunizations, well-care visits, and asthma medications) 
as specified by the HEDIS 2004 criteria were reviewed in the files submitted by Mercy Health Plan 
(see Attachment XIll: Numerator Validation Findings). Medical record reviews were conducted for 


the Adolescent Immunization Status and Adolescent Well-Care Visit measures. 


For the HEDIS 2004 Adolescent Immunization Status Combination #I measure, Mercy Health Plan 
appropriately included administrative events from the State Public Health Immunization Registry 
(MOHSAIC). Mercy Health Plan submitted data files for the Combination #2 Measure and attempts 
were made to validate the Combination #| measure using this file (which is included in the 
Combination #2 measure). A total of 28 unique members were validated in the file, consistent with 
that reported on the DST for the Adolescent Immunization Status Combination #2 Measure. Two 
(2) administrative hits were validated for the Combination # | measure. The dates of birth and 
dates of service were within the valid ranges. There were 62 numerator hits by medical records 
reported. The EQRO sampled 27 of the records for validation. The maximum of 30 was not 
requested, as it was initially assumed the file represented the Combination #1 measure. Given that 
files relating to the Combination #2 measure were submitted, there were only 27 numerator hits by 
medical records reported so the MCO was requested to send all the medical record hits. Nearly all 
(26 of 27) of the medical records requested were received for review, and 24 of them were 


validated by the EQRO. Two records met the criteria for two MMRs by the member’s | 3th 
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birthday, but not the criteria for three Hepatitis B vaccinations. The rate calculated by the EQRO 
based on validated administrative and hybrid hits was 13.87%, an overestimate of approximately 
5.35%. However, it was learned during the on-site interviews that not all medical records for 
members not meeting numerator criteria were requested for review. Mercy Health Plan 
systematically excluded 2! 1 of 394 members from the Adolescent Immunization Status measure 
medical record review by not requesting medical records for members who had no claim in the past 
three years. The total estimated bias as a result of not following the HEDIS 2004 Technical 


Specifications is difficult to determine. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, Mercy Health Plan reported 87 
administrative hits and |5 hybrid hits. In the files submitted, one hybrid hit had a date of service of 
7/26/2002 and was not considered valid for this measure. The medical event codes for 
administrative hits were within valid parameters. The EQRO requested all medical records 
reported as hits from the medical record review (15). Fourteen (14) of the 15 measures were 
received for review, and 4 were validated. When examining the criteria for a complete well-care 
visit, it was found that 6 of the 14 records reviewed showed evidence of a medical history; 6 
showed evidence of anticipatory guidance, and |2 showed evidence of a physical examination. Four 
records met all three criteria to be counted in the numerator. The rate calculated by the EQRO 
based on validated administrative and medical record hits was 22.14%. This results in an estimated 
bias of 2.68% overestimate by Mercy Health Plan. However, the systematic exclusion of 181 of 324 
medical records from the medical record review process makes it difficult to accurately estimate the 


real bias. 


The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was the third 
measure validated. There were a total of unique 271 numerator hits reported and validated. There 
were a total of 1,780 events that were captured for all members. The dates of birth, enrollment and 
services were within the valid ranges. The final rate calculated by the EQRO was 60.63%, with no 


observed bias. 


Sampling Procedures for Hybrid Methods 
The Hybrid Method was used for the Adolescent Immunization Status and the Adolescent Well- 
Care Visits measures. Attachment XIl; Impact of Medical Record Review Findings and Attachment 


XV: Sampling Validation Findings were completed for each of these measures. There were no 
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replacements, contraindications, or exclusions reported by the MCO. All criteria for sampling were 
met for the two measures, except the criteria relating to following specifications regarding the 
treatment of sample exclusions. This rating was based on the exclusion of eligible members from 


the samples. 


Submission of Measures to the State 

Mercy Health Plan submitted the DST for each of the three measures validated to the SPHA, the 
(Missouri Department of Health and Senior Services; DHSS) in accordance with the Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 


Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

Table | shows a summary of the estimated bias in the rates calculated by Mercy Health Plan. Both 
the Adolescent Immunization, Combination #I1 and Adolescent Well-Care Visits measures were 
overestimated, but were within the 95% confidence interval for the rates reported by the MCO. 
The direction and magnitude of total bias, taking into account the systematic exclusion of eligible 


members from the record review is difficult to estimate and unknown. 


Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures. 


Measure Estimate of Bias | Direction of Estimate — 
Adolescent |mmunization Status, Combination #1 5.35% or more Difficult to determine 


Adolescent Well-Care Visits 2.68% or more Difficult to determine 
Use of Appropriate Medication for People with Asthma 0.00% None 





Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 
measure (See Table 2). Although the overestimates found on calculation of the Adolescent 
Immunization Status, Combination #I and the Adolescent Well-Care Visits measures were within 
the 95% confidence interval reported by the MCO, the methods for calculating the measures using 
the Hybrid Method did not follow the HEDIS 2004 Technical Specifications by systematically 
excluding medical records from the review process. As such, they were determined to be Not 


Valid. The Use of Appropriate Medications for People with Asthma measure was Fully Compliant 


and valid. 
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Table 2. Final Audit Rating for HEDIS 2004 Performance Measures. 


Measure mi lateLMcVekeliam.t-halaye 
Adolescent Immunization Status, Combination #1 Not Valid 


Adolescent Well-Care Visits Not Valid 

Use of Appropriate Medication for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by the MCO. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Mercy Health Plan was fully compliant with the specifications for the Use of Appropriate 
Medications for People with Asthma Measure. 


2. Mercy Health Plan used NCQA-certified HEDIS software, CRMS/HPR of McKesson Inc. Most 
of the data for HEDIS measures are stored in the CRMS warehouse from which HPR is used to 
automatically calculate rates. There were clearly defined data fields within the application and it 
automatically feeds the rates as required by the State into the HEDIS Data Submission Tool. 


3. Numerous validation checks in the reporting process, both manual and automatic within the 
software system were conducted. There were sufficient edit checks for data entry. This helps 
in minimizing errors. 


4. There were well-documented policies and procedures for the HEDIS rate calculation measures. 


5. There was food data integration, retrieval and analysis processes in place. This facilitates 
efficient collection of data from claims, credentialing provider, pharmacy, lab, utilization, 
ophthalmic and dental data sources. The State Public Health Immunization Registry (MOHSAIC) 
was used for collection of immunization data which are well-integrated into the system. 


6. There was a focused software development and technical analysis group for data collection, 
programming and analysis. 


AREAS FOR IMPROVEMENT 


|. Mercy Health Plan systematically excluded 211 of 394 members from the HEDIS 2004 
Adolescent Immunization Status measure and |8I of 324 members from the HEDIS 2004 
Adolescent Well-Care Visits measure from the medical record review for the Hybrid Method 
by not requesting medical records for members who did not have a claim in the past three 
years. This is not consistent with HEDIS methodology. 


2. There is an opportunity for better understanding the mechanisms and output results of the 
CRMS/HPR system. There is a need for better ownership and control over the application 
processes. 


3. Data analysis should incorporate tests of statistical significance to assess whether the observed 
changes in rates are related to a specific intervention. 
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RECOMMENDATIONS 


|. Calculate HEDIS performance measures using the Hybrid Method according to HEDIS technical 
specifications or use the Administrative Method for calculation. For the Adolescent 
Immunization Status measure, the Hybrid Method is strongly recommended. 


2. Improve the collection of medical records from providers by stipulating or reinforcing policies 
and procedures referring to medical records storing, archiving and retrieval. 


3. Conduct statistical comparisons on rates from year to year. 


4. |Improve ownership and control by responsible managers of the application to have a better 
understanding of query structures. There is a need for the Mercy Health Plan to have a better 
understanding of the CRMS/HPR system and how this relates to the HEDIS calculation rates. 


5. 


Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data validation in 
the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical Fields? 


For the Medical Claim type, there were a total of 83,489 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.00% complete, accurate, and valid. 
2. The Outpatient Recipient ID field was |100.00% complete, accurate, and valid. 


3. The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.46% valid. 
There were 455 invalid dates of service ranging from |2/23/2002-12/3 1/2003. 


4. The Outpatient Last Date of Service field was 100.00% complete and accurate, 99.00% valid. 
There were 854 invalid dates of service ranging from 4/01/2004-8/2 1/2004. 


5. The Outpatient Units of Service field was 100.00% complete, 99.19% accurate, and 99.19% valid. 
There were 679 invalid entries of “00000.” 


6. The Outpatient Procedure Code field was 100% complete and accurate, and 99.99% valid. 
Invalid procedure codes consisted of 5 ““W0025”, one “Y0025”, and one “Y0029” entries. 


7. The Outpatient Place of Service field was 100.00% complete, accurate, and valid. 


8. The first Diagnosis Code field was 100.00% complete, accurate, and valid. 
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Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second through fifth Diagnosis Code fields were 0.00% complete, accurate, 
and valid. 


For the Dental claim type, there were a total of | 1,273 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 


accurate, and valid. 


For the Home Health claim type, there were no encounter claims paid by the SMA for the period 


January |, 2004 through March 31, 2004. 


For the Inpatient claim type, there were a total of 18,731 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


Se = fs 


The Inpatient Claim Type field was |100.00% complete, accurate, and valid. 

The Recipient ID field was 100.00% complete, accurate, and valid. 

The Admission Type field was 100.00% complete, accurate, and valid 

The Admission Date field was 100.00% complete and accurate, and 95.56% valid. 
There were 831 invalid dates ranging from | 1/30/2003-12/3 1/2003. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 99.07% (with | 74 entries of 
“99999999”. Valid values were present 96.07% of the time. In addition to the invalid 
“99999999” entries, 763 invalid dates ranged from 04/01/2004-07/29/2004. 


The Bill Type field was 100.00% complete, accurate, and valid. 
The Patient Status field was 100.00% complete, accurate, and valid. 


The first Diagnosis Code field was 100.00% complete, accurate, and valid. 


. The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 


completeness, accuracy, and validity established by the SMA (86.07%, 68.20%, 0.00%, and 0.00%, 
respectively). 


. The First Date of Service field was 100.00% complete and accurate, and 95.56% valid. There 


were 831 invalid dates of service ranging from | 1/30/2003- 12/31/2003. 


. The Last Date of Service field was 100.00% complete and accurate, and 95.57% valid. There 


were 829 invalid dates of service ranging from 04/01/2004-07/29/2004. 


. The Revenue Code field was 99.99% complete, accurate, and valid, with one blank field 


(incomplete, inaccurate, and invalid). 


. The Units of Service field was 100.00% complete and accurate, and 99.99% valid, with one blank 


field (incomplete, inaccurate, and invalid). 


Performance Management Solutions Group 211 


a Division of Behavioral Health Concepts, Inc. 


2/16 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 
www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


For the Outpatient Hospital claim type, there were a total of 39,451 encounter claims paid by the 


SMA for the period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.0% complete, accurate, and valid. 
The Recipient ID field was 100.00% complete, accurate, and valid. 

The First Date of Service field was 100.00% complete, accurate, and valid. 
The Last Date of Service field was 100.00% complete, accurate, and valid. 
The Units of Service field was 100.00% complete, accurate, and valid. 


The Revenue Code field was 100% complete, accurate, and valid. 


a 


The first Diagnosis Code field was 99.99% complete, accurate, and valid, with two blank fields 
(incomplete, inaccurate, and invalid). 


8. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell below the 100% threshold established by the SMA for this 
validation. The second Diagnosis Code field was 53.39% complete, accurate, and valid. The 


remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 


was 24.48% complete, accurate, and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
0.00% complete, accurate, and valid. 


For the Pharmacy claim type, there were a total of 72,404 claims paid by the SMA for the period 


January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, accurate, and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Mercy Health Plan, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. There 
were very few errors encountered in the critical fields examined across all claim types. For the 
Medical claim type, the Outpatient Units of Service and Procedure Code fields had invalid entries 
(see above findings); and the Inpatient claim type had invalid Admission and Discharge Date fields. 
There was one blank Revenue Code field in the Inpatient claim type and two bland Diagnosis code 


fields in the first Diagnosis Code field. The Units of Service field had one blank field. 
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lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


To what Extent do the Claims in the State Encounter Claims Database 
reflect the Information Documented in the Medical Record? 

What is the Fault/Match Rate between State Encounter Claims and 
Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 
the period of January |, 2004 through March 31, 2004 for medical record review. Of the 95,566 
Medical encounter claim types in the SMA extract file for January |, 2004 through March 31, 2004, a 
total of 100 encounters were randomly selected. This was an oversample, as the minimum required 
sample size was 88 per MC+ MCO. Providers were requested to submit both medical records and 
claim forms or claim histories for review. There were 64 medical records (64.0%) and 27 claim 
forms (27.0%) submitted for review. Encounters for which no documentation was submitted were 
unable to be validated. The match rate for procedures was 49.0%, with a fault rate of 51.0%. The 


match rate for diagnoses was 49.0%, with a fault rate of 51.0%. 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing information (n = 87) 
or incorrect information (n = 4). For the diagnosis description in the medical record, the reasons 
for diagnoses not matching the SMA extract file were missing or illegible information (n = 56), or no 
match with the description of the symptoms based on the information in the medical record (n = 7). 
For the diagnosis code on the claim form, the reasons for diagnosis codes not matching the SMA 
extract file were missing or illegible information (n = 76), or possible data entry or recording error 
by the provider (n = 2). The reasons for the diagnosis descriptor on the claim form not matching 


the SMA extract file included missing or illegible information (n = 54), and an incorrect code (n = 1). 
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For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = 4), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 2), incorrect codes (n = 7), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = |), illegible information (n = 3), and not enough information to code (n = 3). For the 
procedure code on the claim form, the reasons for procedure codes not matching the SMA extract 
file were missing or illegible information (n = 75), or incorrect code (n = |). For the procedure 
description on the claim form, the reasons for procedure codes not matching the SMA extract file 


were missing or illegible information (n = 88), and incorrect descriptors (n = 1). 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Dental and Pharmacy claim 
types for Mercy Health Plan were consistent with the average for all MC+ MCOs. The Inpatient 
encounter claim type rate was significantly higher than the average for all MC+ MCOs, while the 
rates for Medical and Outpatient Hospital claim types were significantly lower than the average for 
all MC+ MCOs. This could indicate a possible access to care for preventive services that is related 


to a higher need for inpatient services; or may be associated with claims administration. 


The findings from the performance measure analysis of the HEDIS 2004 Adolescent Well-Care Visits 
measure are consistent with the lower rates of Medical and Outpatient Hospital claim types. Mercy 
Health Plan identified significantly fewer administrative hits for the HEDIS 2004 Adolescent Well- 
Care Visits measure than the average for all MC+ MCOs, which again may be associated with claims 
submission or access to preventive care. Mercy Health Plan also identified a significantly lower rate 
of eligible members for the Use of Appropriate Medications for People with Asthma measure, which 


are identified through claims data. 
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What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, Mercy Health Plan had difficulty submitting files in the requested file 
layouts for the encounter data validation and performance measure validation and processes. Files 
were requested in flat file (comma-delimited), machine-readable format, using the SMA layout in 
effect prior to October 2003 for the encounter data analysis. For the performance measure data 
submission, extract files for the HEDIS 2004 Adolescent Immunization Status, Combination #2 
measure were provided rather than the HEDIS 2004 Adolescent Immunization Status, Combination 
#1 measure. This limited the validation of the performance measure. There was no documentation 
associated with any of the files submitted. The following section summarizes the issues found when 


attempting to load files submitted for analysis. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


A letter from Liz Scott dated December 30, 2004 indicates submission of the following three files 
(including the provider file requested for the medical record review, which is not listed). There was 
no other documentation of the files, layouts, definitions, or type of claims submitted (e.g., paid, 


unpaid). There were no unpaid claims submitted for analysis. 


I. Dental.txt 22MB Text Document | 2/29/2004 1:17 PM 
This file was in NSF/CMS 1500 file layout and was able to be loaded. 


2, MCDII229 25 MB File | 2/29/2004 1:43 PM 
This file was not provided in the requested format, but was able to be opened as a text file and appeared to be in the UB-9? file 


layout. There was incomplete data for each record type, but all records shown were able to be loaded for analysis. 


cy MCDP1229 129MB File | 2/29/2004 2:11 PM 
This file appeared to be in the NSF/CMS 1500 file layout. All records provided were able to be loaded for analysis using this 


layout. 
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STRENGTHS 


The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 


The critical fields evaluated for the Dental and Pharmacy claim types were |00.00% complete, 
accurate, and valid. 


Mercy Health Plan demonstrated a rate consistent with the average of all MC+ MCOs of 
validation of the SMA encounter claims extract file against the medical record for procedure 
codes and descriptions. Reasons for medical records not matching were primarily related to 
missing data or illegible information. 


AREAS FOR IMPROVEMENT 


The Outpatient Units of Service and Procedure Code fields in the SMA encounter claims extract 
file for the Medical claim type were 99.19% and 99.99% valid, with invalid entries. 


2. For the Inpatient claim type, the Discharge Date field, Revenue Code field, and Units of Service 
fields were 96.07%, 99.99%, and 99.99% valid, respectively. The Revenue Code and Units of 
Service each contained one blank field. 

3. Mercy Health Plan demonstrated a significantly lower rate than the average of all MC+ MCOs 
for validation of the SMA encounter claims extract file against the medical record for diagnosis 
codes and descriptions. Reasons for medical records not matching were primarily related to 
missing data or illegible information. 

RECOMMENDATIONS 

|. Examine and revise as needed internal system edits for invalid procedure codes in the NSF/CMS 
1500 file layout and run validity checks after the programming of new edits, and ensure 
completion of Units of Service field. 

2. Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 

3. For the Inpatient claim type (UB-92 file layout), improve the rate of valid Discharge Dates to flag 
invalid entries of “99999999"and blank entries for the Units of Service field. Error checks for 
the Diagnosis Code field should also be conducted to ensure no blank fields. 

4. \t is recommended that Mercy Health Plan examine the possible reasons for the lower rates of 
outpatient services (Medical and Outpatient Hospital claim types) and the higher rates of 
inpatient services through a non-clinical performance improvement project aimed at analysis of 
encounter claim and utilization data as well as access to care indicators. 

5. For audit purposes, submit extract files for performance measures and encounter data in the 
requested file layouts with the requested documentation of files. 
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MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described previously in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for Mercy Health Plan (Mercy Health Plan). The EQRO reviewed 
documentation between December |, 2004 and February 28, 2005. On-site review time was used 
to conduct follow-up questions, review additional documentation made available by Mercy Health 
Plan, and provide feedback and recommendations regarding compliance with federal Medicaid 


Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MCO, Mercy Health 
Plan was requested to provide the following documents on-site: 


e Member Handbook 

e Provider Handbook 

e Provider Agreements 

e Grievance and Appeal Policies 

e Samples of both Grievances and Appeals related to member and providers 


Additional documentation Mercy Health Plan included: 


e MC+ Marketing Policy 

e MC+ Marketing Plan 

e Contract Policy Submittal Log 

e Care Issues Report 

e Grievance and Appeals Correspondence 

e Authorization for Disclosure of PHI 

e = Enrollment Policy 

e Emergency and Post Stabilization Care Policy 


Unity Managed Mental Health supplied the following documents regarding their services to Mercy 
Health Plan members: 


e Unity Managed Mental Health Depression Treatment Guidelines 
e ©UMMH ADD Best Practices Screening/Treatment Protocol and Medication Guideline 
e 2004, Quarter 4, Utilization Trend Report 

e 2004 Readmission Report 

e MC+ Most Frequent Diagnosis Report, Quarter 2, 2004 

e 2004 4th Quarter Telephone Indicators Report 

e 2004 Member Satisfaction Report 

e 2004 Provider Satisfaction Report 

e 2004 Provider Satisfaction Survey 

e 2004 2nd Quarter Bipolar Disorder Analysis 

e MC+ Pregnant Women Care Coordination Update 
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e 2004 Network Adequacy Outcomes Report 
e 2004 Provider Record Survey Results 


e Member Transition and Care Policy 


Interviews 


The following individuals were interviewed on March 2, 2005 during the on-site review: 


Plan Administration 

Steve Mead, Product Manager, Medicaid 

Dr. Deborah Zimmerman, Chief Medical Officer 
Pat Snodgrass, Director, Performance 
Improvement 


Quality Assurance 

Steve Mead. Product Manager, Medicaid 

Dr. Deborah Zimmerman, Chief Medical Officer 
Liz Scott, Business Analyst 


Member Services/ Case 

Management/ Utilization Management 

Dr. Deborah Zimmerman, Chief Medical Officer 
Anna Dmuchovsky, V.P., Health Resources 
Susan Meiner, Director, Benefit Coordination 
Donna Hauler, Manager, Member Services 
Debbie Todd, Member Services 

Cindy Johnson, Director, Care Coordination 


Provider Services 

Steve Mead, Product Manager, Medicaid 
Ron Braun, Director, Provider Contracting 
Steve Heinrich, Director, Provider Network 
Management 

Sam Fenner, MC+ Administrator 


Mental Health 

Scott Frederick, PHD, Director, Managed Mental 
Health - UMMH 

Marge Viehland, Manager, Utilization Management 
and Q.|. - UMMH 

Cindy Johnson, Care Coordination 

Liz Scott, Business Analyst 
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FINDINGS 

Enrollee Rights and Protections 

Mercy Health Plan staff were aware of their responsibilities regarding member rights and 
protections. The MCO took pride in its innovative approaches to communicating and sharing 
information with members. The MCO considered itself a leader in providing translated written 
collateral materials, cultural outreach, and production of alternative format materials. The MCO 
Audio Member Guide was produced in four languages. All members received a CD version of their 
Member Handbook. The MCO used a TDD line and Missouri Relay services for the hearing 
impaired and worked with several contractors to provide interpreter services as needed by 
members. Member Services staff had a tenacious approach to contacting members to ensure that 
they had all the information needed to meet their healthcare needs. Provider Services considered 
member issues an essential component of their training curriculum, and retrained on member rights 


whenever a provider’s staff changed. 


Ratings for compliance (15.4%) with regulations concerning Enrollee Rights and Protections were 
the result of a failure to develop and submit MC+ Managed Care Program required policy and 
procedures, and other required documentation to the SMA for their approval. Mercy Health Plan 
had not submitted all required policy. At the time of the on-site review, the MCO did not have an 
approved Member Handbook or Member Marketing and Educational Materials. The MCO provided 
a package that included information and materials that would be valuable to members if these were 
approved and able to be distributed. The MCO had created new documents, “Fast Facts for Adults” 
and “Fast Facts for Children” that were informational brochures that supported PCP advice and 
guidelines for seeking medical attention. None of this material was submitted for the SMA’s review 
or approval. The Member Handbook, also not submitted or approved, included a number of items 
that appeared to contain discrepancies with MC+ Medicaid Managed Care contract required 
language and the federal regulations. The MCO staff did not exhibit a diligent attitude toward 


correcting the problem of timely and accurate submission of policy and materials to the SMA. 


Quality Assessment and Improvement 

Access Standards 

Mercy Health Plan was invested in providing adequate healthcare for their members. They 
maintained an extensive network of PCPs and specialist. The MCO contracted with all regional 


hospitals and both teaching hospitals. They responded immediately if they received a complaint 
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from a member concerning a provider and took action to rectify the issue. The MCO maintained a 
regular schedule of activities to monitor appointment schedules. They used surveys, unannounced 
monitoring of waiting areas, and site visits to ensure that members were treated with respect and 
that providers maintained required contracted standards. This MCO had an extensive network of 
specialists. They negotiated with providers in their commercial network, if these providers did not 
normally see MC+ Managed Care members, whenever necessary. The MCO was willing to pay a 


commercial rate for members who needed specialized care to obtain needed services. 


The MCO had an active case management program. An area of expertise was the coordination of 
care when a member was receiving both behavioral and physical healthcare services. The case 
managers for both the BHO and the MCO used the same authorization system and could view one 
another's entries, including medications prescribed. This method informed staff that member were 
receiving services from more than one system, assisted in sharing information with the PCP and 
mental health provider, and prevented duplication of services or prescribed medications. The 
MCO, BHO, and involved providers have all found this combined system beneficial in planning for 


members’ treatment needs. 


Ratings for compliance with Access Standards (35.3%) reflected problems in the development of 
policy needed to meet the requirements of these regulations. A number of policies reviewed on- 
site, and language in the Member Handbook, contained language that conflicted with the regulations. 
Many policies were not submitted to the SMA for review or approval. To ensure consistent and 


adequate service delivery the issue of policy development and approval must be resolved. 


Structure and Operation Standards 

Mercy Health Plan had an active and thorough credentialing and recredentialing process that 
included mandatory corrective action planning if any problems were identified. The MCO used 
screening guidelines modeled after NCQA standards in qualifying all providers. The practice in this 
area was methodical and systematic and contributed to the assurance that members received quality 


healthcare. 


Ratings for compliance with Structure and Operation Standards ((30%) reflect policy deficiencies 
throughout this section of regulations. The deficiencies in the area of credentialing concerned the 


completion, submission and approval of policy. The MCO did not have completed and approved 
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policy regarding the issue of member disenrollment. The MCO reported that they rarely requested 
member disenrollment from the SMA. The MCO must complete the policy development and 


review process to comply with these regulations. 


Mercy Health Plan shared their system for monitoring the performance of subcontractors with 
reviewers. The reports on monitoring outcomes were not shared and policy development and 
approval was not completed. It was not possible to determine if the MCO met regulation 


requirements regarding subcontractor oversight. 


Measurement and Improvement 

Ratings for compliance with Measurement and Improvement regulations (72.7%) reflected greater 
attention to the details required by this regulatory section. Mercy Health Plan excelled in their 
adoption, dissemination and application of practice guidelines. The MCO had implemented 
preventive health practice guidelines throughout their system, including informing members about 
the services they should expect to receive. The MCO collaborated with other regional MCOs to 
ensure the incorporation of approved national practice guidelines into their program. The MCO 
had a Quality Improvement Committee that reviewed medical records to ensure that these 
guidelines were implemented correctly. The MCO had completed required policy in this area and 


was compliant in the use and implementation of clinical guidelines. 


The MCO had an integral quality assessment and improvement program within their organization. 
The Performance Improvement Department monitored, evaluated, and analyzed all MCO functions. 
This detailed process utilized all information gathered internally, complaints from members, and all 
data possible to inform members and create a substantive quality improvement process. The MCO 
had convened a Medicaid Work Group to review service practices specific to the MC+ Managed 
Care program. The MCO was enlisting members to be part of this group to ensure a holistic 
approach. This level of commitment to quality services was also witnessed in the Validating 
Performance Improvement Projects evaluation. The results of that evaluation can be found in the 


appropriate section of this report. 


The results of the Validating Performance Measures audit can be found in that section of this report. 
The MCO did not follow the HEDIS technical specifications for the calculating performance 


measures resulting in deficiencies. The MCO did have a health information system that was capable 
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of capturing required data. The Validating Encounter Data audit process could not be completed 
because data was not provided in a useful format. These issues reflected problems in validating 
services, medical records, and data that were required for the MCO to reach full compliance with 


the regulations. 


Grievance Systems 

Ratings for compliance with Grievance Systems regulations (100%) indicate that the MCO had 
completed all requirements regarding policy and practice in their grievance and appeal system. Files 
for both grievances and appeals, filed by members and providers, were reviewed on-site. All files 
followed prescribed policy and timelines. Notices to members were sent within required 
timeframes and contained all required information. This information included the member’s ability 
to file a State fair hearing simultaneously with an MCO appeal or later if the outcome of an appeal 
was not favorable to the member. All policy and information to members included the message that 
members can maintain healthcare coverage while an appeal was pending, and explained the 
member’s responsibility to pay for charges if a decision was reached to deny the disputed service. 
The Medical Director was involved in many disputed issues and made every attempt to ensure that 
members obtained the healthcare they needed. If an appeal was filed, it was recorded in the MCO 
tracking system to ensure that another qualified individual reviewed the information submitted to 
obtain an independent appeal decision. The MCO took the grievance and appeal system very 


seriously and used information generated to inform their quality improvement process. 


Summary and Follow-up 

Behavioral Health 

In 2004 it was recommended that UMMH evaluate the use of anti-depressants by PCPs and to 
provide them with the Depression Guidelines. UMMH provided copies of the UMMH Depression 
Treatment Guidelines during the on-site review. They have shared this information with PCPs and 
behavioral health providers. Additionally the case managers from UMMH utilized the same 
authorization system used by Mercy Health Plan. This enabled case managers from both systems to 
update and view information about members who were receiving behavioral and physical health 
services. It also indicated if members were receiving prescriptions from both PCPs and 
psychiatrists. This allowed behavioral and physical health providers to be better informed about 


patients and all the services they received. 
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UMMH shared reports on Utilization Management completed for Mercy Health Plan members. This 
report included utilization objectives, targets and outcomes, and analysis of the data collected. The 
BHO also distributed member surveys evaluating telephone access and member satisfaction with all 
aspects of services. These satisfaction surveys included questions about provider interactions and 
performance of the behavioral health organization. The actions the BHO intended to take during 
2005 included maintaining current performance, improving satisfaction with providers by ensuring 
appropriateness of “provider fit,’ and identifying members with special needs during the intake and 
referral process. The BHO planned to present the results of the 2005 on-site survey to their 


advisory committee for additional action planning. 


UMMH developed ADHD practice guidelines and completed an analysis of Mercy Health Plan 
members presenting with a diagnosis of Bipolar disorder. The BHO presented a report on Mercy 
Health Plan MC+ Pregnant Women Care Coordination. This service was provided to members 
who were receiving prenatal and mental health care. Care coordination protocols were 
implemented by UMMH and Mercy Health Plan in January 2003. The BHO and MCO had access to 
on-line information that allowed for review and integration of member activity within the MCO. 
BHO and MCO case managers used this information in the coordination of care for pregnant 
members with mental and physical health needs. Referrals were made to UMMH for pregnant 
women with substance abuse problems and UMMH notified the MCO of pregnant members who 
came to their attention. The outcome was a 130% improvement in mental health access for 


pregnant women during 2004 over the previous two years. 


Active discharge planning occurred for each member at the time of hospital discharge. In-home 
services were utilized as needed to ensure that the discharge plan was followed and appointments 


were kept. 


To improve access to child psychiatrists, nurse practitioners were added to the network to act as 
physician extenders. Emphasis for development of this resource occurred in the rural areas of the 
MC+ Region. The number of MDs and PhDs improved in 2004. After-hours coverage by providers 


was monitored during 2004 and all providers exceeded the levels of the previous year. 
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Blue Ribbon Physicians’ Network 

This became a quality incentive program meeting all targets set by the MCO. Mercy Health Plan 
promoted the program in the Provider Handbook, and more physicians chose to participate. The 
program doubled over the past year. Mercy Health Plan planned to expand this program to include 


high-volume clinics. 


Reducing Lead Levels 

Mercy Health Plan continued to engage members in lead poisoning prevention through a variety of 
case management activities. The health plan started using Quest Laboratories in an effort to obtain 
test results faster. This change was successful. They utilized this information to set up a case 
management program for individuals identified with high lead levels and were hoping to continue 


influencing this problem in a positive manner. 


Engaging School Districts 

Mercy Health Plan tried to be creative in making connections with school districts to improve 
EPSDT and preventive health treatment to children. They worked with the FQHCs that were 
already connected with schools, and who were providing services on-site. This occurred with 
Peoples’ Health Clinic and the Ferguson-Florissant School Districts, and the St. Louis County Health 
Department and the Jennings School District. Mercy also provided services on-site through the St. 


Louis City Health Department and the Hancock School District. 


Disease Management 
Disease management was provided for members with asthma. Telephonic management was 
provided by Status One, a partner from California. Member data was transmitted to Status One 


where predictive modeling occurred to identify members in need of disease management. 


STRENGTHS 


|. Mercy Health Plan’s partnership with Unity Managed Mental Health (UMMH) strengthened 
their member services in the area of mental and behavioral health, UMMH made their work 
specific to the needs of the MCO’s members. They defined best practices and clinical guidelines 
for issues pertinent to Mercy Health Plan members. They used well-developed clinical guidelines 
that addressed depression management and ADHD. UMMH made a practice of approving and 
using in-home therapy to meet the needs of the member population. They used this service in 
rural areas when members struggle with transportation. They also authorized in-home 
treatment when members (adolescents for example) did not adhere to treatment plans by 
attending office-based sessions. UMMH and Mercy Health Plan shared an authorization system 
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that recorded notes specific to shared members. This system provided accurate information 
regarding diagnosis, treatment, and prescribed medications available to both Mental Health and 
Primary Care Providers. 


2. Provider Relations staff at Mercy Health Plan maintained an up-to-date on-line Provider Manual. 
This manual provided current information to PCPs and specialists. Provider representatives 
visited PCP offices at least quarterly. They also designated a provider outreach position. This 
staff person saw 220 providers specific to the MC+ Managed Care Program. The circuit to visit 
providers was made every 90 to |80 days. During this time, orientation was completed with 
any new office staff member or physicians. The Provider Manual and MCO information was also 
provided in a DVD format for easy reference. 


3. Mercy Health Plan worked with area FQHCs to gain access into school districts to do EPSDT 
examinations. One successful existing partnership included People’s Health Clinic and the 
Jennings-Hancock School District. 


4. The commitment and enthusiasm of the Medical Director and the Members’ Services staff was 
noteworthy. They expressed a sincere desire to improve members’ lives and health. The 
Member Services department continuously strived to create methods to meet the needs of 
members and the community. 


5. Mercy Health Plan convened an MC+ Medicaid Workgroup. This internal workgroup focused on 
assessing current services and developing methods to identify and improve future health care 
initiatives. The MCO was considering including members in this process. 


AREAS FOR IMPROVEMENT 


|. Timely and complete submission of policy and other documentation to the SMA was needed. 
Many of the positive practices that occurred at Mercy Health Plan were not recognized, as they 
were not reflected in current policy or procedural guidelines. 


2. Lack of open communication with the SMA and other entities regarding their MC+ program did 
not encourage confidence in the work produced by Mercy Health Plan. 


3. Lack of written materials that highlight or focus on Mercy Health Plan’s MC+ Managed Care 
Program initiatives created a lack of recognition of the work done in this section of the 
organization. Reports focused on all Mercy Health Plan products. Some of the unique and 
important projects that were specific to MC+ members appeared inconsequential. 


RECOMMENDATIONS 


|. Raise the importance of complying with documentation requirements to the same standards as 
those reflected in the daily practice within the MCO. 


2. Continue MCO development in the area of utilizing available data and member information to 
drive change and create opportunities for organizational growth and development. 


3. Ensure that Mercy Health Plan staff recognizes the importance of complying with both federal 
regulations and the MC+ Medicaid Managed Care contract requirements. 


4. Continue the pro-active approach to tracking and improving services to members. Maintain 
reporting mechanisms to inform both the mental and physical health systems that creating a 
coordinated approach to member services. 
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5. Continue to monitor the ratio of mental health providers to members in rural areas of the MC+ 
Region, particularly in the area of child psychiatry, to ensure adequate access to care. 


6. Continue programs that expand the provider network and its availability to members. 


7. Continue to improve and track the results of case management interventions with members. 
Using available data can provide information to expand or adjust the program to more 
effectively impact individuals who have lead exposure or poisoning. 


8. Continue community-based partnerships as a method of improving the incidence of EPSDT 
examinations and implementing preventive health measures with children/members who may 
otherwise fail to access Mercy Health Plan services. Attempts to expand these partnerships 
were discussed during the on-site review. An expansion would affect members positively. 


9. Continue to collect data from Status One regarding the assessment of member needs through 
predictive modeling and the outcome of coordinated care. The data may provide a basis for 
further study or a best practice presentation. 
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HEALTHCARE USA 


The previous sections of the 2004 EQRO report present the purpose and objectives technical 


methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 


Document Review 


HealthCare USA supplied the following documentation for review: 

Member Reminder Initiative (Well Visits) Performance Improvement Project 
School Based Program Participation 2004 Performance Improvement Project 
School Based Program Communication Form 

School Based Program Power Point Presentation 

HCY/EPSDT Review Tool 

Letter to Provider 


Interviews 

An interview was conducted with the project leader for each of the projects on-site by the project 
director, on Thursday, March 3, 2005 to review the methods, study design, and findings. Technical 
assistance regarding study design and measures was provided as were references for logic model 
development and health services research methods. Specific questions regarding each study were 


addressed: 


Member Reminder Initiative 
e What do the tables at the end mean? 


School Based Program Participation 2004 
e What age groups, grades, schools were the targets of the intervention? 


e How are the forms used in the intervention? As data collection? 
e How did you determine the number referred, screened, consented, and completed? 
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FINDINGS 

The first PIP evaluated was the “Member Reminder Initiative.” The process of accessing or delivering 
care through member reminders for well-care appointments was the topic of the study. It was not 
clear how this issue was identified for improvement. Some background and rationale should be 
presented. An examination of the rate of no-shows for well-care appointment over the past several 
months may have been helpful. This could also be done through identifying current national, state, 


and regional rates for attendance at well appointments. 


Well-care visits for |-month-olds to |2-year-olds were identified as targets for improvement. It was 
assumed that since the MC+ MCO only serves MC+ Members, this was the population at which the 
study was aimed. Some characteristics of the population included or excluded should be presented. 
The study questions were: |) :Does attempting to contact members by phone and/or mail increase 
the number of well-care visits?”; and 2) “Is one method of contacting (telephone vs. mailing) more 
effective? “ The study indicators were HEDIS 2004 and 2005 measures of well-care visits. It was not 
specified whether the Hybrid or Administrative Method for calculation was used. This should be 
specified along with the specific technical specifications (e.g., procedure codes used, age ranges, 
continuous eligibility criteria. VWell-care visits can be linked with improved outcomes. This link and 


rationale should be described in the study topic. 


lt was unclear whether all MC+ Members and groups were included (e.g., 1115, 1915b, children in 
State custody, children enrolled in the consent decree). Although the intervention appeared to 
include all members, the data collection approach was limited to HEDIS eligible members. The 
limitations and possible implications of the criteria for exclusion and inclusion should be described. 
No sampling techniques were described. Although some readers may be familiar with the data 
collection and sources of data for HEDIS measures, this again should be described in a PIP. HEDIS 
measures do provide for consistent accurate data collection over time periods given that the same 
methods of data collection and inclusion and exclusion criteria are used over time. There was a 
good description and the intervention of the processes for improving care. However, there was no 
analysis plan presented and the study questions were not addressed in the data presented. 
Questions that remain include: |) “What was found regarding the relative effectiveness of the vs. 
mail contact with members?”; 2) “Were rates of well-care visits increased?”; 3) “Did they increase 


significantly?”; 4) “Can a significant increase be attributed to the intervention?”; 5) “‘Is it worth the 
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time and effort to contact members?”; and 6) “Are other approaches suggested by the data and the 


implementation of the project?” 


In the presentation of the results, actual rates across time should be presented rather than percent 
change. There was no description of how the percent change was calculated, so it would be difficult 
for one to replicate or understand the method of calculation for future re-measurement. HEDIS 
2004 and 2005 rates were presented. It may be useful to present more than one measurement 
point prior to the intervention to examine trends over time, especially since the measurements are 
annual. Although there was some indication that the mailing was a better mechanism for contacting 
members (a 91.9% contact rate) than telephone (20% contact rate), there was no indication of 
whether the mailing was actually effective in improving the rate of well visits. This could be 
accomplished by examining separately the rate of claims for those who were contacted by the 
telephone relative to those contacted by mail to determine whether either one of these approaches 
actually changed their behavior. It could be that fewer members were able to be contacted by 
telephone, but that this is a more powerful method of getting members to make and attend well- 
care visits than the more passive approach of mailing information. Improvements in processes or 
outcomes of care were difficult to identify based upon the data and manner of presentation. It is 
possible that the change was the result of the intervention. However, a strong case was not 
presented. Comparable time periods were planned for analysis. Since the project was underway, 
there was not a series of repeated measurements to demonstrate effectiveness. Given that 
HealthCare USA has the ability to calculate HEDIS measures in real-time, it is recommended that 
the plan for quarterly measurement be implemented. This study is not likely to result in credible or 


valid findings about the effectiveness of the intervention. 


The effectiveness of the intervention should be discussed in light of the study findings. This would 
address whether the intervention was effective, ineffective, or whether additional data need to be 
collected. The barriers that were identified in implementing the intervention should be used to 
support the findings and identify new components to address in the next implementation cycle. It is 
recommended that the well visits for those contacted by telephone be compared with those 
contacted by mail to address the study question. The statistical significance of change in rates from 


measurement to re-measurement should be examined to substantiate claims of effectiveness. It 
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may be beneficial to report rates for | year and |2-year-olds separately (as the intervention may be 
more effective among different age groups), then use statistical significance testing to determine if 
the intervention was effective and for which ages. The plans for calculating HEDIS measures on at 


least a quarterly basis are strongly encouraged for PIP implementation. 


The second PIP evaluated was, “School-based program participation 2004.” The problem statement 
was, “Because of the general characteristics of HealthCare USA’s membership, all school-age 
children are considered ‘at-risk.’ The reason for problem identification was not provided. The 
intent appears to be aimed at screening and prevention services to provide well-care visits and 
health care screening for “school-aged children.” The study question was clearly stated, “Does 
HealthCare USA, in conjunction with participating schools, increase access to healthcare for ‘at risk’ 


students?” 


It was unclear to what the demographic characteristics of “school-aged” children” referred in terms 
of their age range, type of school (elementary, middle school), or grade levels. Also, the 
presentation subsequently indicates that there were “teens” as the target population. The number 
of students receiving EPSDT screenings and the number of referrals were clearly defined and 
measurable indicators that are associated with health outcomes. It was assumed that all MC+ 
Members were all included and there were no specific groups excluded from the intervention. 
There appeared to be no sampling of members, but rather of schools. It would be important to 
know how and why participating schools were selected, and if and why any may have declined 


participation. 


The actual intervention targeted “all students in designated participating schools that are identified 
as HealthCare USA members and who have not had a claim for a complete physical examination 
within the past year.” This is a good description, but it is important to know in the intervention 
how HealthCare USA members were identified. Part of the intervention was to actually identify 
members at participating schools who were HealthCare USA members who had not received a 
well-care visit in the past year. There was a good description of the intervention and coordination 
with the school-based clinic as well as description of barriers and limitations to implementation and 
plans to modify the intervention in the future. It was not clear how many members would have 
been eligible for screens, how many received parental consent, or how many were ultimately 


screened. This is important to know when interpreting significant or non-significant findings. 
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There were references to several databases, but the only data presented were for the number of 
screens and referrals. It was not clear how the database systems were used to identify eligible 
members and eligible claims, nor was it clear what type of data were in each database. Definitions 
for claims used to identify members (e.g., CPT codes) should be specified. There was no data 
analysis plan described or presented. A second re-measurement point was implied for July 2005, 
with no plan for statistical significance testing presented. Since the study was underway, it was not 


possible to assess whether improvement was real improvement or whether it was sustained. 


The study in question and intervention were well-defined, but the measure of effectiveness is not 
likely to be able to determine whether access to care was increased. Measurements will need to be 


refined and re-calculated to assess effectiveness before any confidence in the findings is developed. 


This PIP is not likely to produce credible findings as currently designed. To clarify the scope of the 
PIP, report the number of students identified the number following up (completing a screening), and 
the proportion by region for each indicator. Baseline and re-measurement should be calculated 
using a percent change score (ending rate — beginning rate/beginning rate) to determine if the stated 
goal of a 50% increase has been approached or attained. A likelihood ratio can also be used to 
calculate the change in actual rates between measurement points. Interventions, selection of 
schools, and barriers in each region should be discussed separately. More frequent measurement 
points (at least quarterly) should be used to assess progress and possible patterns of screening 
during the school year. Given that there is likely some fluctuation in the participation of schools and 
the number of members eligible each re-measurement period, rates and rates of change will need to 


be calculated using raw numbers to assess statistically significant stability or change over time. 


STRENGTHS 


|. There was an excellent choice of study topics. Well-care visits apply to a broad proportion of 
the population and both projects represent aspects of preventive care that can be influenced by 
the MCO. 


2. Findings were disaggregated and presented by region. Given that HealthCare USA operates in 
three regions with distinct populations, provider networks and standard of care characteristics, 
it is important to examine separately the effectiveness of the intervention. 


3. Any differences in the implementation of the intervention and characteristics of the participants 
(e.g., schools) should also be described in detail. 


4. There was good use of standard measures, such as the HEDIS methods for determining access 
to well visits. 
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5. There was inclusion of the entire population of MC+ Members in the interventions and study. 


6. There was excellent crossover of the performance improvement projects with performance 
measures, and good application of the performance measure data. 


AREAS FOR IMPROVEMENT 


|. The study design does not allow for a valid comparison of the effectiveness of the interventions 
on the outcome measures. This would require comparing the two groups over time on the 
outcome measures, not the rate of contact or process measures. 


2. Add more measures of the effectiveness of the implementation (process measures), using 
numerators and denominators as well as proportions for each MC+ Region. 


3. Present data in more detail (provide numbers as well as percents). 
4. Rates should be calculated at baseline and re-measurement, using tests of statistical significance. 


5. Continue with plans to use HEDIS data definitions for continuous measurement and at least 
quarterly re-measurement and interpretation of effectiveness of the intervention. 


6. Compare performance with national and statewide rates for commercial and Medicaid 
Members. 


Validation Of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validating Performance 
Measures Protocol for HealthCare USA. HealthCare USA submitted the requested documents on 
November 23rd, 2004. The EQRO reviewed documentation between November 23rd, 2004 and 
February 28th, 2005. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by HealthCare USA to the SMA 

e The HealthCare USA Baseline Assessment Tool (BAT) for the HEDIS 2004 data reporting year 

e HealthcareData.com LLC’s Compliance Audit Report for HEDIS 2004 

e HealthCare USA’s information systems policies and procedures with regard to calculation of HEDIS 2004 
rates 

e HealthCare USA meeting minutes on information system (IS) policies 

e Asample of Catalyst’s production logs and run controls 

e National Council on Quality Assurance (NCQA)-certified HEDIS software certification report from 
Catalyst Technologies 

e Data field definitions & claims file requirements of the Coventry Corporate Data Warehouse 
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e Data files from the Coventry Corporate Data Warehouse containing the eligible population, numerators 
and denominators for each of the three measures. 

e HEDIS 2004 Data Submission Tool 

e HEDIS 2004 product work plan 


The following are the data files submitted by HealthCare USA for review by the EQRO: 


e ais denominator - Health Care USA central.txt 
e ais denominator - Health Care USA eastern.txt 
e ais denominator - Health Care USA western.txt 
e ais numerator - Health Care USA central.txt 

e ais numerator - Health Care USA eastern.txt 

e ais numerator - Health Care USA western.txt 

e ast denominator - Health Care USA central.txt 
e ast denominator - Health Care USA eastern.txt 
e ast denominator - Health Care USA western.txt 
e ast numerator - Health Care USA central.txt 

e ast numerator - Health Care USA eastern.txt 

e ast numerator - Health Care USA western.txt 

e — field long names.xls 

e HealthCare USA AIS Denominators.txt 

e HealthCare USA AIS Numerators.txt 

e wc3 denominator - Health Care USA central.txt 
e wc3 denominator - Health Care USA eastern.txt 
e wc3 denominator - Health Care USA western.txt 
e wc3 numerator - Health Care USA central.txt 

e wc3 numerator - Health Care USA eastern.txt 
e wc3 numerator - Health Care USA western.txt 


Interviews 

The EQRO conducted on-site interviews at HealthCare USA in St. Louis with Cathie Krueger, Rina 
David-Claytor and Geoff Welsh of HealthCare USA on Thursday, March 3rd, 2005. This group was 
responsible for calculating the HEDIS 2004 performance measures. The objective of the visit was to 
verify the methods and processes behind the calculation of the three HEDIS 2004 performance 


measures. 


FINDINGS 

HealthCare USA calculated each of the three measures validated using the Administrative Method, 
by region of operation (Eastern, Central, and Western). Results were aggregated across all three 
regions for which HealthCare USA reported the measures, as the EQRO is charged with providing 
MCO level comparisons, precluding stratification of analyses by region or other variables. MCO to 


MCO comparisons of the rates of Adolescent Immunization Status Combination #1, Adolescent 
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Well-Care Visits, and Use of Appropriate Medications for People with Asthma measures were 
conducted using two-tailed z-tests. For comparisons that were statistically significant at the 95% 
confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 


significance levels (p < .05) are reported. 


Taking into account the sum of the numerators and the sum of the denominators reported for all 
three regions reported by HealthCare USA to the SMA and the State Public Health Agency (SPHA), 
the rate for the HEDIS 2004 Adolescent Immunization Status, Combination #|1 measure was 4.79%. 
This was significantly lower than the statewide rate for all MC+ MCOs (14.36%; z = -1.05; 95% Cl: 0, 
19.65%; p < .O1). All other MCOs used the Hybrid Method for calculating Adolescent Immunization 
Status, which may have contributed to capturing immunizations that were administered prior to 


1995, when the MC+ Managed Care Program was implemented. 


The rate for HealthCare USA across all three regions for the HEDIS 2004 Adolescent Well-Care 
Visit measure was 31.93%, which was significantly higher than the statewide rate for all MC+ MCOs 
(30.13%; z = 1.32, 95% Cl: 34.43%, 44.19%; p > .95;). The rate for HealthCare USA across all three 
regions for the 2004 HEDIS Use of Appropriate Medications for People with Asthma measure was 
63.32%, which was comparable to the statewide rate for MC+ MCOs (63.92%, z = -2.33; 95% Cr: 
45.65%, 52.93%; n.s.). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 
The information systems management policies and procedures for rate calculation were evaluated 
consistent with the Validating Performance Measures Protocol. This included both manual and 


automatic processes of information collection, storing, analyzing and reporting. 
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For all three measures, HealthCare USA was found to meet all criteria for producing complete and 
accurate data (see Attachment V: Data Integration and Control Findings). There were no biases or 
errors found in the manner in which HealthCare USA transferred data into the repository used for 
calculating the HEDIS 2004 measures. HealthCare USA used an NCQA-certified software vendor, 
Catalyst, for the HEDIS 2004 measure calculation process. The EQRO was provided with a 
demonstration of the Quality Spectrum Hybrid Reporter™, the application module for rate 
calculation, and with Coventry’s corporate data warehouse. The use of an NCQA-certified 
reporter software which has been certified through a process of test files submitted by NCQA 
indicate that the program specifications, codes, and measure parameters are adequate for validly 


reporting the rates. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate (see Attachment VII: Data 
and Processes Used to Calculate and Report Performance Measures). HealthCare USA met nearly 
all criteria that applied for all three measures. The two criteria that were not met involved the use 
of statistical significance testing to document changes in performance over time and using this 
information to support claims of improvement or stability in performance over time. Although 
many auditors conduct statistical tests to examine various aspects of data validity, such tests should 
be used by the MCO to assess the significance of change related to quality improvement activities 
and operational changes. In the future, HealthCare USA plans to use a real-time system to calculate 
HEDIS measures and statistical significance of change over time on at least a quarterly basis for 


performance improvement and monitoring.. 


Processes Used to Produce Denominators 

HealthCare USA met all criteria for the processes employed to produce the denominators of all 
three performance measures (see Attachment X: Denominator Validation Findings). This involves 
the selection of eligible members for the services being measured. Denominators in the final data 
files were consistent with those reported on the DST for all three measures. Across all three 
regions, 5,492 eligible members were reported and validated for the Adolescent Immunization 
Status, Combination #!| measure. There were a total of 9 contraindications identified from 
administrative data for the Adolescent Immunization Status, Combination #| measure across all 
three regions. A total of 41,144 eligible members were reported and validated for the Adolescent 


Well-Care Visits measure across all three regions; and 4,275 eligible members for the denominator 
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of the Use of Appropriate Medications for People with Asthma measure were reported and 
validated across the regions. Age ranges, dates of enrollment, dates of birth, medical events, and 
continuous enrollment were programmed to include only those members who met HEDIS 2004 


criteria. 


Processes Used to Produce Numerators 

All three measures were calculated using the Administrative Method. All three measures included 
the appropriate data ranges for the qualifying events (e.g., immunizations, well-care visits, and asthma 
medications) as specified by the HEDIS 2004 Technical Specifications (see Attachment XIll: 


Numerator Validation Findings). No medical record reviews were conducted or validated. 


HealthCare USA appropriately included administrative events from the State Public Health 
Immunization Registry (MOHSAIC) in calculating the Adolescent Immunization Status measure. For 
the Adolescent Immunization Status, Combination # | measure, a total of 263 administrative hits, 
were reported and validated. The dates of birth range was valid. The dates of service were not 


provided. There was no bias found in the rate calculated by HealthCare USA. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, there were a total of 13,139 
administrative hits. The dates of service were not provided in the numerator files and could not be 


assessed for validity. The dates of birth range was valid. There was no bias found in the rate 


calculated by HealthCare USA. 


The number of administrative hits reported and validated for the HEDIS 2004 Use of Appropriate 
Medications for People with Asthma measure was 2,/07. The codes for numerator events were not 
provided. The dates of birth were in the correct range and valid. The dates of service were not 
provided and could not be validated. There was no bias found in the rate calculated by HealthCare 


USA. 


Sampling Procedures for Hybrid Methods 
No medical record reviews were conducted or validated. Attachment XIl; Impact of Medical 
Record Review Findings and Attachment XV: Sampling Validation Findings do not apply to the 


Administrative Method. 
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Submission of Measures to the State 

HealthCare USA submitted the DST for each of the three measures to the SPHA (the Missouri 
Department of Health and Senior Services) in accordance with the Code of State Regulations (19 
CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality Improvement 
Strategy. 

Determination of Validation Findings and Calculation of Bias 

As referenced earlier, there was no bias found in the reporting of numerators, denominators, or 


rates of the three HEDIS 2004 performance measures validated. 


Final Audit Rating 
The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 


measure. All three measures calculated were Fully Compliant with State specifications. 


STRENGTHS 


|. There were significantly higher rates of HEDIS 2004 Adolescent Well-Care Visit rates compared 
to the average for all MC+ MCOs. This rate was higher than the National Commercial and 
National Medicaid rates for this measure. 


2. HealthCare USA was fully compliant with the calculation of the HEDIS 2005 Use of Appropriate 
Medications for People with Asthma measure. 


3. HealthCare USA employed NCQA-certified HEDIS reporter software, Quality Spectrum ™ of 
Catalyst Technologies. Most of the data for HEDIS measures are stored in the Coventry IDX 
data warehouse from which Quality Spectrum is used to automatically calculate rates. 


4. The Quality Spectrum™ is a very well-designed application based in a SAS environment. The 
application on the whole was very transparent and user friendly. The codes are updated as they 
are released by Catalyst. 


5. The Catalyst application had the ability to provide detailed and in-depth information for “drilling 
down” and evaluation purposes. Even though the codes running behind the program were 
proprietary, the structure of the repository was very informative and provided the interviewers 
with information on event definition, rule definition, flow charts for calculations and had 
metadata definitions. 


6. There were clearly-defined data fields within the application, which automatically feeds the rates 
as required by the State in the HEDIS Data Submission Tool. 


7. There were numerous quality checks in the reporting process done manually with a separate 
abstract of the data files. This helps in minimizing errors. 


8. Efficient data integration, retrieval and analysis processes were in place. Members were 
identified with a unique primary key that avoids duplication errors. Datasets from CareMark, 
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the pharmacy vendor, Quest Labs, the laboratory vendor and the dental vendors were used and 
integrated to Quality Spectrum for calculation of final rates. 


9. There was a good disaster recovery plan in place. 


10. MOHSAIC, the State Public Immunization Registry database, was used for collection of data 
regarding immunizations. The files are received as a text file and are integrated well into the 
system. MOHSAIC data are loaded in a very specific format into the data warehouse. 
MOHSAIC is estimated to account for approximately 14% of the Adolescent Immunization 
Status rate. 


11. HealthCare USA had well-documented procedures for the HEDIS 2004 rate calculation 
measures. 


12. HealthCare USA had a very professional and knowledgeable software management team that 
has an in-depth understanding of the application and knows how to produce accurate results. 


13. Upon review of preliminary validation findings for the performance measure validation, 
HealthCare USA recognized the importance of using the Hybrid Method to calculate the 
Adolescent Immunization Status measure and plans to use this method in the future. 


AREAS FOR IMPROVEMENT 


|. HealthCare USA used the administrative method for the calculation of all HEDIS 2004 
measures. Medical record review is likely to provide unique information regarding adolescent 
immunizations that is not captured in the State Public Immunization Registry for events prior to 
the implementation of the MC+ Managed Care Program and claims systems. 


2. The HEDIS 2004 Adolescent Immunization Status, Combination #! rate was significantly lower 
than the average for all MC+ MCOs. 


RECOMMENDATIONS 


|. The use of the Hybrid Method for the calculation of Adolescent Immunization Status is strongly 
recommended. HealthCare USA should explore the possible reasons for low rates of 
administrative hits for the Adolescent Immunization Status measure. 


2. Integrate HEDIS rate documentation procedures into corporate IS policies. 


3. Data analysis should incorporate tests of statistical significance to assess whether the observed 
changes in rates are related to a specific intervention. 


4. Conduct statistical comparisons on rates from year to year 
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Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data validation in 
the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical Fields? 


For the Medical claim type, there were a total of 445,349 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


The Outpatient Claim Type field was 00.00% complete, accurate and valid. 

2. The Outpatient Recipient ID field was |100.00% complete, accurate and valid. 

3. The Outpatient First Date of Service field was 100.00% complete, accurate and valid. 
4 


The Outpatient Last Date of Service field was 100.00% complete and accurate, and 99.93% valid. 
There were 315 invalid dates of service ranging from 04/01/2004 — 06/29/2004. 


5. The Outpatient Units of Service field was 100.00% complete, accurate and valid. 


6. The Outpatient Procedure Code field was 100.00% complete and accurate, and 94.78% valid 
values. Invalid procedure codes included 23,267 “VWV0026” codes. 


7. The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 14.14% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 6.02% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
2.84% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. 


For the Dental claim type, there were a total of 57,546 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 


accurate and valid. 
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For the Home Health claim type, there were a total of 5 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004. All fields examined except the second through fifth 


Diagnosis Code fields were |00.00% complete, accurate and valid. The second, third, fourth, and 


fifth Diagnosis Code fields fell well below the 100% threshold for completeness, accuracy, and 


validity established by the SMA (0.00% for each). 


For the Inpatient claim type, there were a total of 54,816 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


a ow 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 100.00% complete, accurate and valid. 


The Admission Date field was 100.00% complete and accurate, and 94.57% valid. There were 
2,979 invalid dates ranging from 01/22/2003 — |2/31/2003. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 97.29% (with |,485 entries of 
99999999"). Valid values were present 93.06% of the time. In addition to the invalid 
“99999999” entries, the 2,318 invalid dates ranged from 04/01/2004 — 05/12/2004. 


The Bill Type field was 100.00% complete, accurate and valid. 


The Patient Status field was 100.00% complete and accurate, and 99.89% valid. There were 35 
invalid values of “OO” and 24 invalid values of “63”. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (0.00% for each). 


. The First Date of Service field was 100.00% complete and accurate, and 96.02% valid. There 


were 2,180 invalid dates of service ranging from | 1/16/2003 — 12/31/2003. 


. The Last Date of Service field was |00.00% complete and accurate, and 95.19% valid. There 


were 2,636 invalid dates of service ranging from 04/01/2004 — 05/12/2004. 


. The Revenue Code field was 99.97% complete, accurate, and valid. There were 1/4 invalid blank 


fields. 


. The Units of Service field was 100.00% complete and 99.67% accurate and valid. There were 


179 invalid codes of “OQ0000”’. 


For the Outpatient Hospital claim type, there were a total of 213,448 encounter claims paid by the 
SMA for the period January |, 2004 through March 31, 2004. 


l. 
Z. 


The Outpatient Claim Type field was |00.00% complete, accurate and valid. 


The Recipient ID field was 100.00% complete, accurate and valid. 
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The First Date of Service field was 100.00% complete and accurate, and valid. 

The Last Date of Service field was 100.00% complete and accurate, and valid. 

The Units of Service field was 100.00% complete, accurate and valid. 

The Outpatient Procedure Code field was 100.00% complete, complete, accurate and valid. 


The Outpatient Revenue Code field was 100.00% complete, 63.85% accurate and 63.85% valid. 
Invalid codes were accounted for by 77,167 entries of “000”. 


= oe os 


8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second through fifth Diagnosis Code fields were 0.00% complete, accurate 
and valid. 


10. For the Pharmacy claim type, there were a total of 345,084 claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 
accurate and valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for HealthCare USA, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. There 
were very few errors encountered in the critical fields examined across all claim types. The 
Outpatient Procedure Code field in the Medical claim type included invalid procedure codes (see 
previous findings); while the Inpatient claim type contained invalid data in the Discharge Date and 
Patient Status fields. The Revenue Code field was incomplete, and the Units of Service field 
contained invalid data. For the Outpatient Hospital claim type, the Outpatient Revenue Code field 
contained invalid entries. Although the Procedure Code is optional if the Revenue Code is between 


300 — 319, the Revenue Code is always required?. 


It was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claims extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claims extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


* Personal communication with J udy Muck, Assistant Deputy Director, MC+ Managed Care, April 07, 2005. 
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What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the rate of Outpatient 
Hospital encounter claim types were consistent with the average for all MC+ MCOs. The rates for 
Medical, Dental, Pharmacy, and Inpatient encounter claim types were significantly higher than the 
average for all MC+ MCOs. This suggests high rates of encounter data submission and good access 
to preventive and acute care. 

The findings from the performance measure analysis showed a significantly lower rate of 
administrative hits for the HEDIS 2004 Adolescent Immunization Status, Combination #|1 measure 
and significantly higher rates of administrative hits for the HEDIS 2004 Adolescent Well-Care Visits 
measure than the average for all MC+ MCOs. This may be related to the administration of claims 
for adolescent immunizations. One challenge noted by HealthCare USA was in capturing this data 


through UB-92 claims. 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing information (n = 94) 
or incorrect information (n = |). For the diagnosis description in the medical record, the reasons 
for diagnoses not matching the SMA extract file were missing or illegible information (n = 53), or no 
match with the description of the symptoms based on the information in the medical record (n = 8). 
For the diagnosis code on the claim form, the reason for diagnosis codes not matching the SMA 
extract file was missing or illegible information (n = 80). The reasons for the diagnosis descriptor 
on the claim form not matching the SMA extract file included missing or illegible information (n = 


58), and incorrect codes (n = |). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = 10), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 4), incorrect codes (n = 6), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = 5), illegible information (n = 2) and not enough information to code (n = 3). For the 
procedure code on the claim form, the reasons for procedure codes not matching the SMA extract 


file were missing or illegible information (n = 81), or incorrect codes (n = 3). For the procedure 
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description on the claim form, the reasons for procedure codes not matching the SMA extract file 


was missing or illegible information (n = 86), and incorrect descriptors (n = 2). 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


The only notable issue with the file submission by HealthCare USA was the apparent incomplete 


records on one of the UB-92 file layouts, and the lack of documentation for the unpaid claims files. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


HealthCare USA submitted the following files for encounter data validation. They included paid 
claims for all claim types that were in national standard formats, and unpaid claims. The paid claims 
files were named and labeled such that is was possible to determine the file layout for paid claims. 
The following provides the number of records that were able to be loaded for analysis from each 


file. The unpaid claim files were not associated with any known file layout. 


I. husa.txt 139KB Text Document 10/19/2004 8:55 AM 


This file was in NSF/CMS 1500 file layout and all records were able to be loaded for analysis. 


2. husa0930ub92.txt 8648KB Text Document 10/19/2004 11:02 AM 


This file was in UB-92 file layout and was able to be loaded for analysis, but appeared to have incomplete records. 


3. HcfaFile-EQRO.txt 749KB Text Document | 2/29/2004 10:25 AM 


This file was in NSF/CMS 1500 file layout and all records were able to be loaded for analysis. 
4. PharmacyFile-EQRO.txt 169KB Text Document | 2/30/2004 10:02 AM 


A total of 584 records were loaded for analysis successfully. Some fields contains special and control characters. 


5; UbFile-EQRO.txt 326KB Text Document | 2/28/2004 2:55 PM 
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This file was in UB-92 file layout and was able to be loaded. 


6. UnpaidDentalClaims.txt 592 Text Document | 2/30/2004 12:41 PM 
No files were loaded for analysis, not in NSF/CMS 1500 file layout. 
7. UnpaidHcfaClaims.txt 59KB Text Document | 2/30/2004 |2:42 PM 
No files were loaded for analysis, not in NSF/CMS 1500 file layout. 
8. UnpaidMentalHealthClaims.txt 598 Text Document | 2/30/2004 [2:45 PM 
No files were loaded for analysis, not in NSF/CMS 1500 file layout. 
9. UnpaidUbClaims.txt 28KB Text Document | 2/30/2004 12:42 PM 
No files were loaded for analysis, not in UB-92 layout. 
STRENGTHS 
|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 
2. The critical fields examined for the Medical, Dental, Home Health, and Pharmacy claim types 
were |100.00% complete, accurate and valid. 
3. There were significantly higher rates of claims for preventive and acute care services than the 
average for all MC+ MCOs. 
4. There were significantly higher rates of administrative hits for the HEDIS 2004 Adolescent Well- 


Care Visits measure than the average for all MC+ MCOs. 


AREAS FOR IMPROVEMENT 


For the Medical claim type, there were invalid Outpatient Procedure Codes. 


For the Inpatient claim type, there were invalid entries for the Discharge Date and Patient 
Status, and Units of Service fields, and blank Revenue Code fields. The Revenue Code is a 
required field regardless of the Procedure Code. 
For the Outpatient Hospital claim type, there were invalid data in the Outpatient Revenue Code 
field. 
The match rate for procedures and diagnoses between the SMA encounter claims extract file 
and the medical records for HealthCare USA were significantly lower than the average for all 
MC+ MCOs. The majority of errors were due to missing or illegible information, and was 
accounted for primarily by the submission of medical records by providers. 
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RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the NSF/CMS 
1500 file layout for the Outpatient Procedure Code and run validity checks after the 
programming of new edits. 


2. Ensure that Discharge Date, Patient Status, and Units of Service fields are complete and valid for 
the Inpatient (UB-92) claim types, and institute error checks to identify invalid data. 


3. Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 


MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described previously in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for HealthCare USA (HealthCare USA). The EQRO reviewed 
documentation between December |, 2004 and February 28, 2005. On-site review time was used 
to conduct follow-up questions, review additional documentation made available by HealthCare 
USA, and provide feedback and recommendations regarding compliance with federal Medicaid 


Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MC+ MCO, 
HealthCare USA was requested to provide the following documents on-site: 


e Member Handbook 

e Provider Handbook 

e Provider Contracts 

e Grievance and Appeal Policies 

e Samples of both Grievances and Appeals related to member and providers 
e Credentialing Policy 


Additional documentation made available by HealthCare USA: 


e Marketing Plan and Educational Material Development Policy 
e HealthCare USA of Missouri Organizational Chart 

e NICU Focus Study 

e Denial Log 

e HealthCare USA Policy Manuals 
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Interviews 


The following individuals were interviewed on March 3, 2005 during the on-site review: 


Plan Administration 

Claudia Bjerre, President and CEO 
Mary Mason, MD, Medical Director 
Jackie Inglis, QI Director 


Quality Assurance 
Jackie Inglis, QI Director 
Mary Mason, MD, Medical Director 


Member Services/ Case 

Management/ Utilization Management 
James Howarth, Director of Operations 
Melody Lyles, Manager of Service Operations 
Kim Martin, Service Center Trainer 

Linda Whaley, VP of Health Services 

Laura Ferguson, Director of Health Services 
Mary Mason, MD, Medical Director 

Tiffany Smith, Manager of Compliance 

Pam Luebbert, Director of Community 
Development 

Melanie Fahrner, Manager of Community 
Development 


FINDINGS 


Provider Services 

Bonnie Kitson, VP of Provider Affairs 
Resmi J acob-Schrieber, Director of Network 
Operations 

Kara House, Supervisor, QI 

Leslie Sisavath, Credentialing Analyst 


Mental Health 

Jerry Matthews, PhD, VP Regional Operations, 
MHNet 

Susan Norris, MHNet 

Cynthia Williams, MHNet 

Kara House, Supervisor, QI 


Enrollee Rights and Protections 

HealthCare USA maintained a strong commitment to member rights. The MCO’s focus on treating 
members with respect and dignity was pervasive throughout the organization. Cultural diversity was 
an essential component to interactions with members. There was a company wide training initiative 
on cultural diversity to educate all staff on sensitivity to cultural issues in communications and 
decision-making. All interviews provided evidence that respect for members was foremost in staff 
thinking and their approach to work. HealthCare USA had staff who spoke different languages, they 
maintained an up-to-date list of providers who spoke a language other than English, and they had 
their written materials translated into Spanish and Bosnian. The MCO administrative staff remained 
in touch with the issues and problems encountered at all levels throughout the organization. They 


assisted in problem solving on member issues and were invested in positive outcomes for members. 


Ratings of compliance with Enrollee Rights and Protections (38.5%) reflected required MC+ 
Managed Care Program policy that was not complete. Interviews with staff indicated an awareness 


of the need to be vigilant about the production of policy and the annual review process. When 
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questions were raised about outstanding policy the MCO, in many cases, produced revised policy 
ready to be submitted to the SMA for their final review and approval. The MCO was encouraged to 
complete this process to ensure compliance with the MC+ Medicaid Managed Care contract, as well 


as federal regulations. 


Quality Assessment and Performance Improvement 

Access Standards 

HealthCare USA had a number of programs in place that were a benefit to members. They had a 
large provider network that included a close working relationship with Washington University 
School of Medicine. They had an in-depth maternity program that ensured proper care for 
members, as well as in-home follow-up for mom and baby for all deliveries. The MCO had a strong 
case management program that included special needs outreach coordinators to put members with 
special healthcare needs in touch with additional community resources that exceeded those covered 
by the healthcare benefit. These coordinators also assisted members who needed a specialist as a 
PCP when this was in the member’s best interest. The MCO Provider Relations staff included an 
educational component in the work they did in provider offices to ensure proper identification of 
members with special healthcare needs. They believed this early identification was essential to 


facilitate timely intervention and access to all appropriate services. 


Ratings of compliance with Access Standards regulations (52.9%) reflected a number of HealthCare 
USA policies that required submission to the SMA for final approval. The rating indicated a finding 
of several instances of conflicting information in the Member Handbook and the Provider Resource 
Guide. These revisions were essential not only for compliance with all federal regulations, but also 


for the consistent delivery of an effective program. 


Structure and Operation Standards 

The MCO operated an effective Provider Relations program. They had approved credentialing 
standards. The MCO also provided on-going training and monitoring to ensure that their provider 
network maintained required standards. Performance for both contractors and subcontractors was 
monitored through office or on-site visits, educational programs, prompt response and resolution of 


complaints, and consumer surveys. 
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Ratings for compliance with Structure and Operations Standards (60%) reflected a lack of approved 
policy and procedures regarding member disenrollment. This policy is required to comply with the 


MC+ Medicaid Managed Care contract and the federal regulations. 


Measurement and Improvement 

HealthCare USA used InterQual criteria for utilization management. The MCO adopted practice 
guidelines for diabetes based on the ADA criteria, AAP guidelines for prescribing Synagis, and 
ACOG guidelines for neonatal care. The MCO had entered into an agreement with Washington 
University to study asthma in children. The University had a disease management program for 
asthma and the MCO planned to implement their practice guidelines. The MCO described meeting 
with providers to educate them regarding the implementation and use of practice guidelines. 
HealthCare USA offered Continuing Medical Education (CME) credit for attendance, but it was not 
clear how many physicians attended these sessions. The MCO did not have a clear method to 


ensure that these guidelines are uniformly applied. 


The MCO demonstrated its commitment to its Quality Assessment and Improvement Program 
through its policy and practice. HealthCare USA shared their Quality Management Committee 


Charter, and the processes this group used to meet its goals. 


The MCO did appropriately report, through the administrative method, data required for Validating 
Performance Measures. Additional work was identified by the audit for Validating Performance 
Improvement Projects. The details are located in the appropriate sections of this report. The 
MCO did have a health information system (HIS). The validation of encounter data was not 
completed as the data was not submitted in a usable format. The details of this issue are located in 


the Validating Encounter Data section of this report. 


Ratings for compliance with Measurement and Improvement regulations (63.6%) reflect the lack of 


policy and appropriate data validation. 


Grievance Systems 
Ratings for compliance with Grievance Systems regulations (100%) indicate that the MCO had 
completed all requirements regarding policy and practice in their grievance and appeal system. Files 


for both grievances and appeals, filed by members and providers, were reviewed on-site. All files 
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followed prescribed policy and timelines. Notices to members were sent within required 
timeframes and contained all required information. This information included the member’s ability 
to file a State Fair Hearing simultaneously with an MCO appeal, or later if the outcome of an appeal 
was not favorable to the member. All policy and information to members included the message that 
members can maintain healthcare coverage while an appeal was pending, and explained the 
member’s responsibility to pay for charges if a decision was reached to deny the disputed service. 
The Medical Director was involved in many disputed issues and made every attempt to ensure that 
members obtained the healthcare they needed. If an appeal was filed, it was recorded in the MCO 
tracking system to ensure that another qualified individual reviewed the information submitted to 
obtain an independent appeal decision. The MCO took the grievance and appeal system very 


seriously and used information generated to inform their quality improvement process. 


Summary and Follow-up 

Behavioral Health 

MHNet implemented practice guidelines in 2004 for major depression, schizophrenia, bi-polar 
disorder for adults and children, substance abuse, and ADHD. Provider practice was measured 


using these guidelines and feedback was given through provider profiles. 


MHNet implemented two performance improvement projects. One project focused on improved 
provider adherence to practice guidelines for bipolar disorder and major depression. A second 
performance improvement project focused on the impact of preventive initiatives for members with 
ADHD. Both projects were specific to the MC+ Managed Care Program population. The goal of 
these projects was to lead to improved intervention methods used by the BHO to be more effective 
for MC+ Managed Care members. To date the BHO reported improved use of guidelines by 


providers and continued use of available therapy by members diagnosed with ADHD. 


MHNet continued to work on a number of other follow-up issues. Using a case manager assigned 
to outreach services, the BHO was intervening sooner with members discharged from in-patient 
treatment. Hospitalized members were contacted to learn if they had a current therapist. If they 
did, the therapist was contacted and informed about the hospitalization. The BHO requested 
follow-up appointments for the member. If the member did not have a current provider, the 
member was contacted prior to release from the hospital and provided the names of at least three 


(3) outpatient therapists for follow-up treatment. The case manager asked the member to make the 
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first outpatient appointment within three days of discharge from any hospitalization. If no 
appointment was made, and the BHO knew the provider of choice, the provider was contacted and 
asked to do additional follow-up. This encouraged members to keep appointments and obtain 
effective follow-up care, which decreased the need for in-patient episodes. 

Availability of psychiatrists continued to be a struggle for the BHO. One of the methods used to 
influence the lack of psychiatric availability was contracting with Advance Practice Nurses. This 
extended the availability of psychiatric services, particularly in the Central MC+ Region. Another 
positive for HealthCare USA was contracting with the new Midwest Behavioral System, which 
resulted from the merger of Royal Oaks and Pathways. Use of this group increased the entire 


behavioral health network and the availability of psychiatrists in the Central MC+ Region. 


Upon member enrollment the MCO notified MHNet of any identified mental health needs. MHNet 
notified HealthCare USA when they learned that a member was pregnant to ensure that appropriate 
prenatal services were in place. There was also a system of coordination in place with the pharmacy 


provider, who notified MHNet if there was an apparent substance abuse issue. 


NICU Focus Study 

HealthCare USA conducted a focus study to assist in determining potential causes for the increasing 
number of low birth-weight infants. They considered evidence from national studies and the 
application of this information to local issues. HealthCare USA began their study by examining all 
NICU admissions during 2003, and included information on the average length of stay. They 
reviewed and analyzed the reasons for admission, and summarized the mother’s demographic data. 
HealthCare USA attempted to administer a Prenatal Risk Assessment on new members coming into 
the health plan when they were pregnant. They did the assessment on all members that could be 
contacted. Contacts included attempted visits by an in-home health agency. Information on all 
members who could be reached was analyzed for the types and amount of prenatal care received, 


and risk factors indicated. 


This study was not complete at the time of the on-site review. Analysis was continuing on 
interventions that were aimed at achieving demonstrable improvement, subsequent or modified 


interventions for improvement, sustained improvement, and lessons learned. 
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One intervention that was having a positive impact on reducing the number of low birth weight 
infants was the use of the drug PI7. This drug was being administered to a group of women 
through a study at Washington University. These women previously delivered premature and low 
birth weight infants, and were included in the study regardless of risk factors. Though the study was 
in its early stages, the results to date were remarkable for reducing premature delivery. Most of the 
infants born after the administration of this drug were in the normal birth weight range and did not 


require NICU admission. 


HealthCare USA reported that the combination of the completion of the original study, conducting 
the Prenatal Risk Assessment, intense in-home case management interventions, and the inclusion of 
high-risk mothers in the P| 7 study, had the potential to identify a means of significantly reducing the 


number of low birth weight infants. 


Member Compliance and Satisfaction 

HealthCare USA implemented a number of practices to increase information to new members, and 
to improve member satisfaction. One noteworthy area was the periodic inclusion of management 
staff on the telephone banks. Management staff intermittently make Welcome Calls and answer the 
member services toll free information line. This activity provided insight into the difficulties MCO 
staff had in contacting new members based on the enrollment information received, and on issues 
that members were bringing to member services. Management staff report that in a number of 
instances problems could be fixed very quickly. HealthCare USA should utilize the information they 
gain from these activities to drive changes in organizational communication. If problems were 
identified by staff, and could be remedied by a “quick fix,” there could be a method to communicate 
this to management on a regular and timely basis. This would enable the MCO to continue to be 


more responsive to member issues. 


Healthcare USA initiated a project called “Provider Language Data Collection 2004,” in an effort to 
identify alternative languages spoken by providers and office staff. This study was completed by the 
Provider Relations and Outreach Departments. They contacted all PCPs and high volume specialists 


to obtain information regarding languages spoken by the provider, and office staff. The information 
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obtained was entered into the Coventry Provider Database (CPD). Language updates occur at the 


time of credentialing and/or recredentialing. In the third quarter of 2004 Provider Relations 


developed a report for Member Services containing all of the providers who spoke a language other 


than English. This report was updated and distributed to Member Services on a quarterly basis. 


Completing a formal study on the number of members who come into the MCO with correct 


contact information, and the time and resources expended on correcting addresses and telephone 


numbers was considered. The MCO viewed this as a serious problem that they had not been able 


to resolve. 


STRENGTHS 


Integration of all specialty sections within the HealthCare USA organization enhanced the 
service delivery system. Individuals interviewed could not only speak to their specific role at 
HealthCare USA, but how their work affected that of other sections. The responses to 
interview questions demonstrated how staff knew one another’s focus and projects to ensure a 
holistic approach to service delivery. 


MCO staff exhibited a commitment to excellence in service delivery throughout their 
organization. Staff at all levels demonstrated a desire to ensure that the healthcare services 
provided met the needs of all members and were of high quality. There was a common 
expectation that members be treated with respect and dignity. 


HealthCare USA was working with Washington University to develop innovative approaches to 
solving problems and providing new alternatives to serve members. Two specific initiatives 
were mentioned. There was a project linking members who had children with autism to 
community resources, as treatment for autism was not a covered benefit. Another project 
utilized the drug PI 7 to prevent premature deliveries. 


MCO staff exhibited a significant recognition of the regional and cultural differences in the areas 
HealthCare USA served. The MCO recognized the need for diversity within the organization, 
and in approaches to the different communities where services were delivered. 


HealthCare USA required that all managers provide services throughout different levels within 
the organization. The CEO, Vice Presidents, and Directors all spent time on the pre- 
certification and member services telephone lines each quarter. They reviewed cases quarterly, 
and participated in other activities to ensure a true understanding of members’ issues and the 
way services were delivered at all levels within their organization. 


AREAS FOR IMPROVEMENT 


HealthCare USA needed to track and complete their documentation requirements to ensure 
that policies and procedures were submitted to the SMA in a timely manner. This would ensure 
that the MCO was compliant with the MC+ Medicaid Managed Care contract and the federal 
regulations. 
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RECOMMENDATIONS 


Raise the importance of complying with documentation requirements to the same standards as 
those reflected in the daily practice and operations within the MCO. 


Continue MCO development in the area of utilizing available data and member information to 
drive change and create opportunities for organizational growth and development. 


HealthCare USA should work with MHNet to ensure that the outcomes of their performance 
improvement projects are shared with PCPs and with MCO staff. Positive outcomes may 
encourage more effective communication between BHO providers and PCPs. These projects 
can also positively influence the treatment methods utilized for members with complex issues. 
Monitoring and using information gained may be beneficial to the MCO and to MC+ Managed 
Care members. 


Continue the BHO practice of contacting and providing follow-up case management services to 
members leaving in-patient hospitalization. Ensure that statistics are collected and analyzed to 
provide data on the outcome of these interventions. Continue expanding the network and using 
creative methods to ensure that members have adequate access to specialties such as 
psychiatrists. 


The MCO should review BHO case management processes annually as part of subcontractor 
oversight. The review should include an audit of case management records for documentation 
of measurable and updated objective treatment plans, activity toward treatment goals, and 
follow-up of hospitalization with outpatient treatment. 


Share objective study information with the SMA and ask them to become a partner in problem 
solving regarding member contact information. This is a common concern throughout all of the 
MCOs. Having well-researched statistics to share may increase the opportunity for effective 
problem solving with the SMA. 


The MCO is encouraged to continue the adoption of clinical guidelines. Further, the MCO is 
encouraged to include the provider's use of guidelines in the assessment of their practice for the 
purpose of provider profiling. 
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The previous sections of the 2004 EQRO report present the purpose and objectives technical 


methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 
Documents Reviewed 


Missouri Care supplied the following documentation for review: 


e Increasing Appropriate Childhood Immunization Practices 
e Increasing Blood Lead Testing by Primary Care Providers 


Interviews 

Project Leaders, participants, and the Medical Director were interviewed on-site by the EQRO 
Project Director on Wednesday, March 23, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided as were references for logic 
model development and health services research methods. Specific questions regarding each study 


were addressed: 


Childhood Immunization Practices 

e Why was the intervention limited to members meeting HEDIS eligibility criteria? 

e In what instances would improved documentation in the medical record NOT contribute to higher rates of 
immunizations? 

Why was the study population limited to only those who had immunizations? 

Is this a longitudinal study on a specific cohort (N = 943 member who had immunizations)? 

What are statistical process control methods and how have these been used? 

What would be a significant difference at the 95% confidence level? 

Has a project been implemented during 2004? 

Data analysis began in January 2005. What were the findings? 

How will the HEDIS Childhood Immunization Rates be measured? What is the definition of the measure? 
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Blood Lead Testing 
e How many providers were visited in July and August 2004, and what proportion was successfully educated? 


e How does the study question relate to the hypotheses and measures? 

e Data for the previous 12 months and next 12 months will be collected. How frequently, and for which 
measures? 

e How many PCPs and how many members? 

e If the rates will be measured against a 10% increase and against the state rate of 75%, what is the purpose of 
using measures of central tendency and standard deviations? 

e How is the number of member months used in the calculation of rates? Describe the calculation of the 
indicators. 


FINDINGS 

The first PIP evaluated was, “Increasing Appropriate Childhood Immunization Practices.” The topic 
or problem statement was, “Although the Missouri Care Childhood Immunization rate of 64% in 
2004 was well above the state average for Missouri Medicaid of 47%, the health plan failed to meet 
the NCQA HEDIS 75th percentile benchmark of 68%”. There was a very clearly written section 
describing the rationale, the implications for immunizations in improving health status and presenting 
national, state, and MCO rates, as well as barriers encountered and goals. The topic addressed a 
broad spectrum of key aspects of enrollee care and services. The study question was, “Does 
improved documentation in the medical record increase the percentage of children who meet 
immunization periodicity requirements by their second birthday?” The intervention involved 
sending a letter and sample immunization report to providers. The study and intervention 
appeared to exclude those with no immunizations documented in the State Public Health 
Immunization Registry (MOHSAIC) or Missouri Care’s claim system, as well as those who did not 
meet HEDIS enrollment criteria. The study question was clear but implied that the main purpose 
is to place documents in the medical record to improve HEDIS reporting, rather than impacting the 
access, timeliness, or quality of care. A better question might be, “Does informing and reminding of 
providers of the immunization status of members improve access to and timeliness of the 
administration of vaccinations by the member’s second birthday?” There was reference in the study 
topic section to racial and economic disparities however, it was not clear whether all racial and 
ethnic groups were included, or whether the findings were specific to MC+ members. Immunization 


status is clearly associated with long-term health outcomes which should be clearly stated. 


The annual measure of Childhood Immunization Status, Combination #1 was used to assess the 
effectiveness of the intervention. Although this is a standard measure with detailed specifications, 
the method of calculation (Administrative or Hybrid), the data sources (e.g., medical record, 


MOHSAIC, claims data), and the specifications of numerators and denominators (e.g., CPT 
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procedure codes) should be defined. The data collection approach excluded those who may not 
have met HEDIS criteria for enrollment, as well as those who did not have immunizations. All 
eligible members should have been included according to HEDIS technical specifications. It was not 
clear whether sampling was conducted as there was no indication of whether or not the 
Administrative or Hybrid Method would be employed for calculating the HEDIS measure. There 
were references to quarterly data collection for monitoring and statistical process control methods, 
but the data to be collected was not specified. HEDIS measures are typically calculated on an annual 
basis. The HEDIS Combination #| measures were well specified in the HEDIS Technical 
Specifications manual. If followed properly, the data would be reliable and valid however, this was 
difficult to assess based on the information provided. The screenshot of the Missouri Care database 
used appeared to allow for accurate data collection. It would be important to specify in written 
instructions and training how the users are to enter data and use the database. The 95% confidence 
intervals and target rates should be identified based on information presented and available baseline 
rate. Sources of data appear to be member, claim, and provider. There was no indication of what 
indices to which the statistical process control methods would be applied. Initial and repeat 
measurements were identified as 2004 and 2005. It was unclear how or what quarterly data were 
going to be examined or analyzed. Given that there were no re-measurement points at the time of 


validation, it was not possible to assess whether there was real or sustained improvement. 


Measurement of the success of the implementation of the intervention was not conducted. The 
intervention involved sending a letter and sample immunization report to providers. However, it 
was not possible to identify how successful the intervention was in getting a report to providers. 
Other sources for intervention were implied, but not specifically identified for future improvement 


(e.g., educating about billing to improve documentation). 


The presentation of data should be consistent for each data point. Figures should stand alone. 
Indicate in footnotes what Combination #| measure might mean. Numerators and denominators 
should be used with an indication of statistical significance of changes. There was good identification 
of one limitation which should be explained further with an assessment of the degree to which this 


may affect study findings. 
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The study has potential for credible findings with major revisions. This is a good non-clinical 
intervention with the potential to improve medical record documentation of immunizations. There 
is a risk that the annual measurement of one indicator will not be sensitive enough to show the 
impact of the intervention. The study is limited to only those with an immunization documented 
who meet HEDIS eligibility criteria, potentially excluding members transferred from other MC+ 
MCOs or those not having had any immunizations. The intervention does not appear to be strong 
enough to impact changes in quarterly measures or HEDIS rates. Statistical significance testing will 


be important in making any conclusions about the impact of any interventions. 


In the future, describe the methods and procedures for data collection, and details of measurement. 
Add short- and intermediate-term indicators of the intervention. Annual measurement will take a 
several years of intervention to show improvement, thus, quarterly re-measurement is 
recommended. Expand the intervention and study to include all children 2 years of age and under, 


not just those meeting HEDIS criteria and having immunizations. 


The second PIP validated was, “Increasing Blood Lead Testing by PCPs.” The topic statement was 
that “Missouri Care data demonstrates that testing in both the |2-month and 24-month age group 
falls far below the benchmark at 8% and 5%, respectively. This was a good description and rationale 
for study and quality improvement. The study question was, “Will the provider intervention 
increase the percentage of children tested for lead at 12 and 24 months of age?” Additional 
constructs were implied and measured, including the rate of missed opportunities (a measure of 
access) and the rate of members requiring hospitalization for chelation therapy. The barriers 
identified included the ability to conduct blood testing in provider offices and parental refusal for 
testing. It appeared that all providers were included in the intervention and all |2- and 24-month- 
olds were included in the study. The rate of blood lead testing is strongly associated with long-term 


health outcomes. 


The study used well-operationalized indicators specifying sources and codes, but the time frame of 
measurement (the range before and after the | 2th or 24th month birthday for which an 
immunization could be counted) could be more clear. The best indicators were the rate of blood 
lead testing, the percent of missed opportunities, and the percent of members hospitalized for 
chelation therapy. The rationale and application of data points |2 months prior to and |2 months 


following the intervention as well as the calculation of measures using member months was 
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confusing. Increased knowledge of providers and satisfaction with training were mentioned but not 
included in the list of indicators. This would be good process measure of the effectiveness of the 
implementation of the intervention. Provider knowledge, attitudes and beliefs were measured 
through a survey but were not identified as indicators or study measures and no survey was 
provided for a review. Three indicators were chosen and one was related to the hypotheses. Data 
sources were clearly identified (providers, claims data). In order to document validity and reliability 
of data collection over time, more specific operationalization of how claims data were extracted 
would improve the ability to consistently measure the indicators over time. Also, there was 
confusion about measurement points and methods of calculating rates. These methods will need to 
be specific in order to ensure the same data are collected over time. The data analysis plan referred 
to statistical process control charts, measures of central tendency, and standard deviations as well as 
comparisons with State benchmarks and designated percent increase in the target rates. It was not 
clear how these would be applied to the measures or interpretation of findings to determine 


success of the intervention. 


In terms of whether all MC+ Members were included, the age of members was defined and the 
MC+ population was assumed, but additional information regarding the characteristics of the study 
population (MC+ members only) should be provided. In terms of the data collection approach, all 
PCPs were included in a survey, but it was not clear how many PCPs or which PCPs (e.g., internal 
medicine, family practitioners, OB/GYNs, etc) were included. No sampling was conducted. In 
presenting the intervention and the scope of the intervention, there should be a description of 
training given to providers as well as the persons targeted and trained in provider offices (e.g., 
attending physicians, residents, office managers, clerks). Also, the number of providers or provider 
offices actually trained out of the number targeted should be presented as a process measure of the 
scope of the intervention. The number given a survey and the response rates should always be 
presented, along with the results of the survey. The analysis should involve addressing questions 
such as; |) “Were there enough physicians reached to have an impact on the outcomes?”; 2) “Were 
physicians satisfied with the training?”; 3) “Did their knowledge increase?”; 4) “Did they increase 
their use of verbal lead screening forms and filter paper testing?” The data for one of the three 
indicators was presented. Given that data collection was retrospective and there were two 
quarters since the intervention, the other two indicators could have been examined on a quarterly 


re-measurement cycle. 


Performance Management Solutions Group 258 
BHC a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Presentation of the data could be more clear. Line graphs depicted |2 and 24-month-old claims 
percentages but did not indicate the number of points of intervention or which figures were pre and 
post-intervention. There was mention of use of a statistical quality control chart and descriptive 
Statistics but no upper and lower confidence intervals were provided. Time periods of 
measurement were one year prior to and one year post-intervention. There was no discussion of 
the effectiveness of the intervention or methods of improving the intervention based on the 
preliminary findings. Given that there were no re-measurement points at the time of validation, it 
was not possible to assess whether there was real or sustained improvement. 

This was a generally well thought-out and designed improvement project with a reasonable 
intervention. It should be continued and refined, as there is moderate potential for improving the 
process and outcomes of care if the intervention is strong enough and the measures can be 
evaluated in a reliable and valid manner. The data analysis can be improved by specifying the data 
extraction parameters and using descriptive statistics. The use of member months as a denominator 


was not clear especially since data are presented in percents (rate per |00). 


Missouri Care should evaluate all the indicators specified in the PIP and evaluate the implementation 
of the intervention and assess whether it is strong enough or whether additional steps need to be 
taken to address specific barriers. Use process measures of the effectiveness of the implementation 


of the intervention, clarify methods of calculating measures, and develop a valid analytical plan. 


STRENGTHS 


|. Identification of the problem and rationale for the study topic were well written and addressed a 
broad spectrum of members. 


2. The hypotheses and study questions suggested a number of measures. The “missed 
Opportunities” measure was a unique one that can be used for feedback to providers. 


AREAS FOR IMPROVEMENT 


|. The studies as written were difficult to validate and there was little institutional memory for the 
processes due to turnover in the quality management department. This highlights the 
importance of clearly defined, written, and documented studies as well as policies, procedures, 
and protocols. 


2. Measures of the number of members in the target population that were reached by the 
intervention and other measures indicating they were reached are important in evaluating the 
implementation of the intervention. 


3. Numerators and denominators of measures should be clearly defined and described; and 
calculated consistent with the definitions. 
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Prior to the implementation of the PIP, there should be a clear plan for analysis (e.g., statistics 
used to compare pre- post- intervention measures, analyses for each measure, and re- 
measurement time periods). Modifications in the plan at the time of analysis should be 
explained. 


Consider whether the intervention(s) chosen are likely to impact the measures used, and 
whether the interventions were strong enough to produce the intended outcomes. Measures 
or interventions may need to be adjusted accordingly. 


RECOMMENDATIONS 


Use logic models to develop study questions, hypotheses, and measures. The study question, 
hypotheses, measures, and interventions should be logically linked to one another. 


Use process measures (short-term measures) to assess the effectiveness of the implementation 
of the intervention (e.g., number and rate of physicians contacting the MCOs regarding 
immunizations, rate of use of lead screening and filter paper forms for blood lead testing, 
evaluation of the training). 


Identify and describe measures and calculations in detail. Describe the formulas and definitions 
used for calculating numerators and denominators and use these in the presentation of data. 


Continue to attend to changes in measurement and adjust accordingly. When the calculation of 
measures changes, this needs to be clearly pointed out in any verbal or graphical presentation of 
analyses. Ideally, data should be re-calculated retrospectively to ensure continuity with novel 
definitions. 


Consider all interventions that may be impacting the intermediate and long-term outcomes. 
Document all interventions and system changes that may have an impact (positive or negative) 
on the findings, outcomes, or interpretation of the study. 


Validation Of Performance Measures 


This section describes the documents, data, and persons interviewed for the Validation of 


Performance Measures for Missouri Care. Missouri Care submitted the requested documents on 


November 23rd, 2004. The EQRO reviewed documentation between November 23rd, 2004 and 


February 28th, 2005. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


METHODS 


Objectives, technical methods, and procedures are described under separate cover. This section 


describes the documents, data, and persons interviewed for the Validation of Performance Measures 


for Missouri Care (Missouri Care). Missouri Care submitted the requested documents on 


November 23rd, 2004. The EQRO reviewed documentation between November 23rd, 2004 and 
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February 28th, 2005. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by Missouri Care 
e The Baseline Assessment Tool (BAT) submitted by Missouri Care 

e MEDSTAT’s NCQA HEDIS Compliance Audit Report for 2004 

e HEDIS 2003 custom code mapping 

e HEDIS 2003 APS Data repository, Repository File Structure 

e Missouri Care’s HEDIS Data Entry Training Manual 

e Missouri Care’s Policies pertaining to HEDIS rate calculation and reporting 


The following are the data files submitted for review by the EQRO: 


e Medications Use for Asthma Sample.xls 
e Adolescent well care sample.xls 
e Adolescent Imm Sample.xls 


Interviews 

The EQRO conducted on-site interviews with Brenda Moore, Katie Dunne, and via conference call 
with Greg Cohen (of Austin Provider Solutions), Karen Richards and Jean Vertefeville (NCQA- 
certified auditors) at Missouri Care in Columbia on Wednesday, March 23rd, 2005. This group was 
partly responsible for the process of calculating the HEDIS 2004 performance measures. There was 
recent turnover in staff at Missouri Care responsible for coordinating the calculation of performance 
measure. The objective of the on-site visit was to verify the methods and processes behind the 
calculation of the three HEDIS performance measures. This included both manual and automatic 


processes of information collection, storing, analyzing and reporting. 


FINDINGS 

Missouri Care calculated the Adolescent Immunization Status, Combination #] and Adolescent 
Well-Care Visits measures using the Hybrid Method. MCO to MCO comparisons of the rates of 
Adolescent Immunization Status Combination #1, Adolescent Well-Care Visits, and Use of 
Appropriate Medications for People with Asthma measures were conducted using two-tailed z-tests. 
For comparisons that were statistically significant at the 95% confidence interval (Cl), the z-score 


(z), the upper and lower confidence intervals (Cl), and the significance levels (p < .05) are reported. 
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The HEDIS 2004 rate for Adolescent Immunization Status, Combination #I| reported to the SMA 
and the State Public Health Agency (SPHA) by Missouri Care was 52.07%. This was significantly 
higher than the statewide rate for all MC+ MCOs (14.36%; z = 1.15; 95% Cl: 37.21%, 66.93%; p > 
95). 

The HEDIS 2004 rate for Missouri Care for the Adolescent Well-Care Visits measure was 41.49%, 
which was significantly higher than the statewide rate for all MC+ MCOs (30.13%; z = .1.61, 95% Cl: 
36.61%, 46.37%; p > .95). The 2004 HEDIS rate for Missouri Care for the Use of Appropriate 
Medications for People with Asthma was 65.41%, which was significantly higher than the statewide 


rate for MC+ MCOs (63.92%, z = .56; 95% Cl: 61.77%, 69.05%; p > .95). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

The information systems (IS) management policies and procedures for rate calculation were 
evaluated consistent with the Validating Performance Measures Protocol. For all three measures, 
Missouri Care was found to meet all criteria for producing complete and accurate data (see 
Attachment V: Data Integration and Control Findings). There were no biases or errors found in 
the manner in which Missouri Care transferred data into the repository used for calculating the 


HEDIS 2004 measures. 


Documentation of Data and Processes 

Missouri Care and its affiliate, Schaller Anderson, contracted with Austin Provider Solutions (APS) 
for the calculation of the HEDIS 2004 performance measures. APS used an internally-developed 
application for the measures to be calculated. The application is in the final stages of NCQA 
certification and has been reviewed by the NCQA-certified auditor, MEDSTAT, for source code 
validation and efficiency of data integration. The EQRO was provided with a process overview of 
the QMACS claims management system, a registered trademark owned by Quality Care Solutions, 
Inc. (QCSI), and a validation overview of the HEDIS Data repository of APS. The EQRO was also 


provided with an overview of the data flow and integration mechanisms for external databases for 
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these measures. Data and processes used for the calculation of measures were adequate (see 
Attachment VII: Data and Processes Used to Calculate and Report Performance Measures). 
Missouri Care met nearly all criteria that applied for all three measures. Two of the criteria that 
were not met involved the use of statistical significance testing to document changes in performance 
over time and using this information to support claims of improvement or stability in performance 
over time. Although many auditors conduct statistical tests to examine various aspects of data 
validity, such tests should be used by the MCO to assess the significance of change related to quality 
improvement activities and operational changes. One item that was not met was related to the 
implementation of the medical record review process. Missouri Care was unable to locate the data 
sheets with member identifying information and medical record review findings from the HEDIS 


2004 review. 


Processes Used to Produce Denominators 

Missouri Care met all criteria for the processes employed to produce the denominators of all three 
performance measures (see Attachment X: Denominator Validation Findings). This involves the 
selection of members eligible for the services being measured. 

Missouri Care employed a 5% oversample rate for the Adolescent Immunization Status and 
Adolescent Well-Care Visits measures. For the Adolescent Immunization Status measure, there 
were two records excluded due to contraindications identified through administrative data, and 
there were two records chosen from the auxiliary list for replacement, making for a total sample of 


All. 


For the HEDIS 2004 Adolescent Immunization Status, Combination #] measure, there were a total 
of 41 | administrative numerators in the validated file. There was no duplication of members. The 
dates of birth were within the valid range. The dates of enrollment were not provided for 
validation. The total number of administrative hits reported was 63, and 61 were validated by the 


EQRO. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, there were a total of 41 | members 
listed in the files provided by Missouri Care. The DST shows a denominator of 388, with a final 
sample size of 408 after a 5% oversample. There were no exclusions allowed for the measure, and 
no exclusions or replacements were reported. There was no explanation provided for the 


observed discrepancy. There were no duplicate member names, identification numbers or dates of 
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birth. The dates of birth were within the valid range. The dates of enrollment were not provided. 


Codes for well care visits were not provided. 


For the HEDIS 2004 Use of Appropriate Medications for People with Asthma measure, there were a 
total of 344 denominators reported and validated. There were no duplicate members and the dates 


of birth were in the valid range. The dates of enrollment were not provided. 


Processes Used to Produce Numerators 

All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2004 criteria (see 
Attachment XIll: Numerator Validation Findings). Medical record reviews were conducted for the 


Adolescent Immunization Status and Adolescent Well-Care Visit measures. 


For the HEDIS 2004 Adolescent Immunization Status Combination #I measure, Missouri Care 
appropriately included administrative events from the State Public Health Immunization Registry 
(MOHSAIC). A total of 61 of the 63 administrative hits were validated. For the medical record 
review validation, the EQRO requested 30 of the 15! medical records reported to have contributed 
to the hybrid hits. A total of 23 of the 30 medical records were received for review, and |5 of them 
were validated by the EQRO. The rate calculated by the EQRO based on validated administrative 


and hybrid hits was 33.09%, resulting in an overestimate of 18.98%. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, |24 of the 125 administrative hits 
reported were validated. The EQRO requested a sample of 30 of the 36 medical records reported 
to have contributed to the hybrid hits. Twenty-four (24) records were received for review and 2 
were validated. This resulted in a rate of 32.47%, a 9.02% overestimate. When examining the 
criteria for a complete well-care visit, it was found that 7 of the 23 records reviewed showed 
evidence of a medical history, |7 showed evidence of a physical examination, and 8 showed evidence 
of anticipatory guidance. Two records met all three criteria to be counted in the numerator. 

The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was the third 
measure validated. All 225 numerator hits reported were validated. The dates of services range for 
administrative hits were within the valid range. Codes for numerator events were not available and 


could not be validated. There was no bias observed. 
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Sampling Procedures for Hybrid Methods 
The Hybrid Method was used for the Adolescent Immunization Status and Adolescent Well-Care 
Visits measures. Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: 


Sampling Validation Findings were completed for each of these measures. 


Submission of Measures to the State 

Missouri Care submitted the DST for each of the three measures validated to the SPHA (the 
Missouri Department of Health and Senior Services; DHSS) in accordance with the Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 


Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

Table | shows the estimated bias and the direction of bias found by the EQRO. Both measures 
calculated by the Hybrid Method were beyond the 95% lower confidence limits reported by the 
MCO. There was no bias observed in calculation of the Use of Appropriate Medications for People 


with Asthma measure. 


Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures 


Measure Estimate of Bias Direction of Estimate 
Adolescent |mmunization Status, Combination #1 Overestimate 


Adolescent Well-Care Visits Overestimate 
Use of Appropriate Medication for People with Asthma None 





Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 
measure. Table 2 below summarizes Final Audit Ratings based on the Attachments and validation of 


numerators and denominators. 


Table 2. Final Audit Rating for HEDIS 2004 Performance Measures 


Measure mi latelMcVereliam»t-halaye 
Adolescent |mmunization Status, Combination #1 Not Valid 


Adolescent Well-Care Visits Not Valid 

Use of Appropriate Medication for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by Missouri Care. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 
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STRENGTHS 


Missouri Care was fully compliant with the HEDIS 2004 Use of Appropriate Medications for 
People with Asthma measure. 


Missouri Care demonstrated a significantly higher rate of the HEDIS 2004 Use of Appropriate 
Medications for People with Asthma measure compared to the average for all MC+ MCOs, and 
the National Commercial rate. 


There was effective use of data from the State Public Health Immunization Registry (MOHSAIC) 
for the calculation of the immunization rates. Austin Provider Solutions (APS) provides the 
members’ identification data to the SPHA and obtains an extract of immunizations for the 
members eligible. This data are loaded into the HEDIS data repository (SQL Server) and 
integrated into the calculation of the numerators. 


Missouri Care used reporting software by APS for calculation of HEDIS performance measures 
that has been reviewed by the NCQA-certified auditor, MEDSTAT. This application software 
has been reviewed and tested for source code verification by MEDSTAT, but is not NCQA- 
certified or tested. 


Missouri Care had no capitated providers, which may have contributed to higher rates of 
administrative hits. 


There were effective edit and validity checks within the QMACS claims system. There was a 
mechanism to create error logs within QMACS which enables the user to create reports and 
verify accuracy. 


Missouri Care has identified several actions for improving the HEDIS rate calculation process in 
the future, based on feedback provided at the exit interview and preliminary performance 
validation findings. These include closer coordination with APS, documentation of the process, 
improved security of data, the use of statistical significance testing comparing rates from year to 
year, support APS’ pursuit of NCQA certification for software, and additional staff training for 
HEDIS measure calculation. 


AREAS FOR IMPROVEMENT 


There is a need for improved documentation on the HEDIS rate calculation process. Missouri 
Care should maintain HEDIS project plans and polices related to the calculation of the HEDIS 
measures on-site. 


There is an opportunity to increase ownership and control by Missouri Care over the rate 
calculation process. There is a need to have adequate knowledge transfer processes in the 
event of turnover within the organization. Due to recent turnover, Missouri Care had no “pull 
sheets” with medical record review data for individual patients. 


Missouri Care used a medical record review software, which was used as a tool for retrieving 
medical records that needed to be reviewed. This application has a module for data entry that 
does not indicate whether administrative data has been previously entered. The design of this 
module and the requirement to re-enter data leaves open the possibility for data entry error of 
valid administrative data. Missouri Care follows the process identified by the NCQA auditor, 
MEDSTAT. 


Data analysis should incorporate tests of statistical significance to assess whether the observed 
changes in rates are related to a specific intervention. Currently, as per NCQA guidelines, 
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auditors monitor the percentile change in annual rates, but there was no indication whether any 
Statistical tests of significance were conducted on the rates or communicated to Missouri Care; 
or whether such tests were used by Missouri Care to examine the significance of increases or 
decreases from year to year. 


RECOMMENDATIONS 


Improve documentation of the HEDIS rate calculation process by developing and maintaining a 
set of information system policies for the HEDIS rate production on-site to allow for continuity 
and validity of the process of rate production in the event of turnovers. 


Increase ownership and control by Missouri Care over the process of calculation of HEDIS 
measures by designating an employee to closely coordinate with APS and clearly assigning 
responsibilities related to the HEDIS rate calculation processes within the organization. 


Review privacy and security policies with regard to medical record handling and storage of 
protected health information (per the Health Insurance Portability and Accountability Act) 
within the organization. 


Conduct and document statistical comparisons on rates from year to year. 


Encourage APS to pursue the NCQA certification for the HEDIS performance measure rate 
reporting software. 


Training of MCO staff involved in the oversight of coordination of performance measure 
calculation is strongly recommended. The NCQA offers workshops on the calculation of 
HEDIS measures. 


Validation of Encounter Data 


FINDINGS 


The findings for the encounter data validation are organized according to the encounter data 


evaluation questions presented in the Technical Methods section for the encounter data validation in 


the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical Fields? 


For the Medical claim type, there were a total of 93,315 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


l. 
2. 


The Outpatient Claim Type field was |100.00% complete, accurate and valid. 


The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

The Outpatient First Date of Service field was 100.00% complete, accurate and valid (with 
rounding). Three invalid dates of service ranged from 12/09/2001 — [2/23/2003 for three fields. 
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4. The Outpatient Last Date of Service field was 100.00% complete, accurate and valid (with 
rounding). Four invalid dates of service ranged from 04/08/2004 — 06/24/2004. 


The Outpatient Units of Service field was |00.00% complete, accurate and valid. 
The Outpatient Procedure Code field was 100.00% complete, accurate and valid. 
The Outpatient Place of Service field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


eS oo = 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 47.48% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 21.88% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
9.41% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Dental claim type, there were a total of 9,956 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 


accurate and valid. 


For the Home Health claim type, there were no encounter claims paid by the SMA for the period 


January |, 2004 through March 31, 2004. 


For the Inpatient claim type, there were a total of 7,326 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 


The Admission Type field was 100.00% complete, accurate and valid. 


-~ @ 


The Admission Date field was 100.00% complete and accurate, 94.46% valid. Invalid dates 
ranged from I 1/11/2003 — 12/31/2003. 


5. The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 99.82% of the time (with I3 entries 
of “99999999"). Valid values were present 97.86% of the time. In addition to the invalid 
“99999999” entries, invalid dates ranged from 04/01/2004 — 04/02/2004. 


6. The Bill Type field was 100.00% complete, accurate and valid. 


7. The Patient Status field was 100.00% complete with the correct number of characters (size). 
The correct type of information and valid codes occurred 99.89% of the time. There were eight 
invalid codes of “16”. 
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8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (85.59%, 63.21%, 0.00%, and 0.00%, 
respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 94.46% valid. Invalid 
dates of service ranged from I 1/11/2003 — 12/31/2003. 


ll. The Last Date of Service field was 100.00% complete and accurate, and 98.03% valid. There 
were four invalid dates of service ranging from 04/01/2004 — 04/02/2004. 


12. The Revenue Code field was 99.99% complete, accurate, and valid. One invalid field was blank 
(incomplete, inaccurate, and invalid). 


13. The Units of Service field was 100.00% complete and 98.98% accurate and valid. Invalid values 
were accounted for by 75 entries of “QO000”. 


For the Outpatient Hospital claim type, there were a total of 50,707 encounter claims paid by the 
SMA for the period January |, 2004 through March 31, 2004. Missouri Care had 100.00% complete, 
accurate and valid data for all fields examined, except the third through fifth Diagnosis Codes. 
Although the second through fifth Diagnosis Code fields are optional according to the Health Plan 
Record Layout Manual, they fell well below the 100% threshold established by the SMA for this 
validation. The second Diagnosis Code field was 51.51% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field was 
23.44% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fourth and fifth Diagnosis Code fields were 0.00% 


complete, accurate and valid. 


For the Pharmacy claim type, there were a total of 70,403 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. Missouri Care had 100.00% complete, accurate and valid 


data for all fields examined. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Missouri Care, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. All critical 
fields for the Medical, Outpatient Hospital, Dental and Pharmacy claim types were |00.00% 


complete, accurate, and valid (see previous findings). The Inpatient Claim type had invalid data in 
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the Discharge Date, Patient Status, and Units of Service fields, with one blank field for the Revenue 


code. 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 
selected by the EQRO for validation. The following section summarizes the findings of the data 


submission process. 


What is the Level of Volume and Consistency of Service? 

When comparing the rate of encounter claim types per 1,000 members, the rates for Dental and 
Inpatient encounter claim types were consistent with the average for all MC+ MCOs. The rates for 
Medical, Pharmacy, and Outpatient Hospital claim types were significantly higher than the average 
for MC+ MCOs. This suggests high rates of encounter data submission and at least moderate 


access to preventive and acute care. 


The findings from the performance measure analysis were consistent with the high rates of 
encounter claims, as Missouri Care identified significantly higher administrative hits for the HEDIS 
2004 Adolescent Immunization Status, Combination #1, Adolescent Well-Care Visits, and Use of 


Appropriate Medications for People with Asthma measures. 


To what Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? 

What is the Fault/Match Rate between State encounter Claims and 
Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 


the period of January |, 2004 through March 31, 2004 for medical record review. 


Of the 95,566 Medical encounter claim types in the SMA extract file for January |, 2004 through 
March 31, 2004, a total of 100 encounters were randomly selected. This was an oversample, as the 


minimum required sample size was 88 per MC+ MCO. Providers were requested to submit both 
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medical records and claim forms or claim histories for review. There were 6/7 medical records 
(67.0%) and 21 claim forms (21.0%) submitted for review. Encounters for which no documentation 
was submitted were unable to be validated. The match rate for procedures was 44.0%, with a fault 


rate of 56.0%. The match rate for diagnoses was 50.0%, with a fault rate of 50.0%. 


What Types of Errors Were Noted? 


An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing information (n = 90) 
or incorrect information (n = 3). For the diagnosis description in the medical record, the reasons 
for diagnoses not matching the SMA extract file were missing or illegible information (n = 37), or no 
match with the description of the symptoms based on the information in the medical record (n = 

| 1). For the diagnosis code on the claim form, the reasons for diagnosis codes not matching the 
SMA extract file were missing or illegible information (n = 81), or possible data entry or recording 
error by the provider (n = 2). The reasons for the diagnosis descriptor on the claim form not 


matching the SMA extract file included missing or illegible information (n = 74), and incorrect codes 


(n = I). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = 6), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 5), incorrect codes (n = 5), and 
upcoding (billing a service that is reimbursed at a higher rate or for more time than actually spent 
with the patient; n = 9). For the procedure code on the claim form, the reasons for procedure 
codes not matching the SMA extract file were missing or illegible information (n = 77), or an 
incorrect code (n = 3). For the procedure description on the claim form, the reasons for 
procedure codes not matching the SMA extract file were missing or illegible information (n = 85), 


and incorrect descriptors (n = 1). 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 


SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
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for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, Missouri Care had difficulty submitting files in the requested file layouts 
for the performance measure validation and encounter data validation processes. Files were 

requested in flat file (comma-delimited), machine-readable format. Microsoft Access and Excel files 
were submitted in response to the request for file layouts in national standard formats. This limited 
the ability conduct validation of encounter claims data. The following section summarizes the issues 


found when attempting to load files submitted for encounter data validation analysis. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


There was no documentation, file layouts, or programs associated with the file submission from 


Missouri Care. 


I. EPSDT 4 10.mdb 2,/96KB Microsoft Access Application [2/27/2004 3:14PM 
This Microsoft Access file consisted of two tables, with indicators for type of form (“1500” and “UB-92) and a field 
indicating “paid” and “unpaid”. There were no records able to be loaded for analysis, as it was not in any known file 


layout and there was no documentation provided. 


2. EQRO List 2004.xls 153KB Microsoft Excel Worksheet 12/23/2004 3:27PM 
This Microsoft Access file consisted of two tables, with indicators for type of form (“1500” and “UB-92) and a field 
indicating “paid” and “unpaid”. There were no records able to be loaded for analysis, as it was not in any known file 


layout and there was no documentation provided. 


3. EQRO TbI.xls 513KB Microsoft Excel Worksheet 12/27/2004 3:12 PM 
This Microsoft Excel file consisted of two tables, with indicators for type of form (“1500” and “UB-92) and a field 


indicating “paid” and “unpaid”. There were no records able to be loaded for analysis. No documentation provided 


4. 04122301 .hcf 2,868KB —HCF File [2/23/2004 3:00 PM 
This file was able to be loaded for analysis using the NSF/CMS 1500 file layout. 


5. 04122301.hip 57KB HIP File [2/23/2004 3:03 PM 


This file was able to be loaded for analysis using the UB-92 file layout. 
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6. 04122301.hop 207KB HOP File [2/23/2004 3:15 PM 


This file was able to be loaded for analysis using the UB-92 file layout. 


STRENGTHS 


|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 


2. The critical fields examined for the Medical, Dental, Outpatient Hospital, and Pharmacy claim 
types were |100.00% complete, accurate and valid. 


3. The rates for Medical, Pharmacy, and Outpatient Hospital claim types were significantly higher 
than the average for MC+ MCOs, suggesting high rates of encounter data submission and at 
least moderate access to preventive and acute care. 


4. The rates of identification of administrative hits for performance measures were significantly 
higher for Missouri Care, suggesting more complete claims data for immunizations and well-care 
visits. 


5. Missouri Care had significantly higher rates of match for the procedure and diagnosis codes 
between medical records and the SMA encounter claims data than the average for all MC+ 
MCOs. This was largely accounted for by the higher rate of medical records submitted by 
providers, as errors were primarily related to missing or illegible records. 


AREAS FOR IMPROVEMENT 


|. The Discharge Date, Patient Status, and Units of Service fields had invalid entries for the 
Inpatient claim type. 


RECOMMENDATIONS 


|. Ensure that Discharge Date, Patient Status, and Units of Service fields are complete and valid for 
the Inpatient (UB-92) claim types, and institute error checks to identify invalid data. 


2. Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 


3. For audit purposes, submit extract files for performance measures and encounter data in the 
requested file layouts with the requested documentation of files. 
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MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described previously in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for Missouri Care (Missouri Care). The EQRO reviewed 
documentation between December |, 2004 and February 28, 2005. On-site review time was used 
to conduct follow-up questions, review additional documentation made available by Missouri Care, 
and provide feedback and recommendations regarding compliance with federal Medicaid Managed 


Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MC+ MCO, Missouri 
Care was requested to provide the following documents on-site: 


e Member Handbook 

e Provider Handbook 

e Provider Agreements 

e Member Marketing and Education Plan 

e Sample of Grievance and Appeal files representing both members and providers 


Additional documentation provided by Missouri Care on-site: 


e Appointment Standards and Travel Distance Policy 

e Member Rights and Responsibilities Policy 

e Record Retention Policy 

e Quality Management Plan Policy 

e MC+ Marketing Materials 

e Missouri Care Health Plan Highlights Calendar Year 2004 

e Filter Paper Lead Screening Tool Kit for providers 

e Lead Screening Information for Schools and Families 

e Missouri Care Newsletters for Member, Providers, School Nurses 
e Missouri Care New Member Packet 

e Marketing Material brochures and school information 

e Grievance and Appeal — Compliance Procedure ( 07/07/04) 

e Provider Complaints Procedure/Remittance Advice Message language 
e Service Investment Committee Meeting Minutes - 2004 

e Medical Quality Management Meeting Minutes — 2004 

e Credentialing Committee Meeting Minutes — 2004 

e Record Retention Policy 

e Prior Authorization Policy 

e = Schaller Anderson of Missouri — Organizational Chart 
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Interviews 


The following individuals were interviewed on March 23, 2005 during the on-site review: 


Plan Administration 

Susan Christy, Executive Director and Plan 
Administrator 

Jan Swaney, MD, Chief Medical Officer 

Brenda Moore, RN, Manager, Medical Management 


Quality Assurance 

Brenda Moore, RN, Manager, Medical Management 
Jan Swaney, MD, Chief Medical Officer 
Tammy Wiese, Manager, Quality Management 
Debbie Langley, Member Solutions Manager, 
Grievance Manager 

Katie Dunne, Senior Quality Management 
Coordinator 

Jean Gurucharri, RN, Quality Coordinator 
Katie Dunne, Senior Quality Management 
Coordinator 

Jean Gurucharri, RN, Quality Coordinator 


FINDINGS 


Enrollee Rights and Protections 


Member Services/ Case 

Management/ Utilization Management 
Debbie Langley, Member Solutions Manager 
Gina Cooper, RN, Prior Authorization 
Marilyn Thomas, RN, Case Manager 

Robin Hubble, RN, Case Manager 

Marilyn Frazier, RN, Case Manager 


Provider Services 

Susan Christy, Executive Director, Plan 
Administrator 

Catherine Noel, Manager, Provider Relations 


Mental Health 

Brenda Moore, RN, Manager, Medical Management 
Melody Dowling, UM Manager 

Jan Swaney, Chief Medical Officer 


By Teleconference: 
Carol Matyas, VP Public Sector, Schaller-Anderson 
Corporate Office 


Missouri Care exhibited a strong commitment toward member rights. The MCO staff explained 


that they believed in the inherent dignity of each member they serve. Cultural competency training 


was regularly scheduled for all employees in an effort to educate and raise awareness of cultural 


issues. Missouri care used the AT & T language line to ensure that interpreter services were always 


available. They used a TTY line to communicate with members who were hearing-impaired. They 


had the capacity to have materials put in alternative formats for members who were visually 


impaired or had limited reading proficiency, as requested. The MCO staff used their initial contacts 


with members to identify conditions that would create barriers for members in understanding or 


using the services available. Missouri Care was committed to assisting members in resolving these 


issues. 


Ratings for compliance with Enrollee Rights and Protections ((53.8%) reflected a number of policies 


that Missouri Care must complete and submit to the SMA for review and final approval. The MCO 


was aware of the policy requirements and expressed a willingness to correct this issue. 
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Quality Assessment and Performance Improvement 

Access Standards 

Missouri Care, through its association with the University of Missouri, University Physicians 
Network, had access to PCPs and specialists throughout the MC+ Managed Care Region. The 
efforts of the MCO ensured that there were healthcare services in areas that were previously 
underserved, particularly for the MC+ Managed Care population. The MCO was willing to utilize 
out-of-network providers when this would best serve a member’s needs. They did not find that the 
need for out-of-network providers occurred regularly due to the extensive network available. 
Provider Services staff monitored the issue of timely access by making regular visits to provider 
offices, and responding to member complaints. When a problem was identified, corrective action 
was implemented quickly. The MCO took additional steps such as identifying providers, including 
dentists, who set aside blocks of time to serve members, particularly on an emergency or urgent 
care basis, and encouraged providers to send out reminder notices for preventive care 


appointments. 


Missouri Care had an extensive case management program that provided assessment of all members 
identified with special healthcare needs. The MCO ensured that these members had access to all 
required services. The case management team worked with members to overcome barriers they 
might have when attempting to obtain healthcare. This staff was knowledgeable about the MCO 
benefits and about available community resources that might assist members. The case managers 
utilized the assistance and knowledge of local health departments in accessing some services and to 
ensure that members’ needs were met. The case managers admitted that creativity and diligence 
were required due to the variety of issues they encountered throughout their eighteen county MC+ 


Region. 


The ratings for compliance with the Access Standard regulations ((70.6%) reflected that the MCO 
continued to have a number of policies requiring completion and submission to the SMA for their 
review and approval. A number of policies were returned to the MCO, by the SMA, for revision. 


These revisions were not re-submitted to the SMA at the time of the on-site review. 


Structure and Operation Standards 
Ratings for compliance with the Structure and Operation Standards regulations ((90%) reflect that 


the MCO was substantially compliant with this section of regulations. They had approved policy and 
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sound practice regarding credentialing and recredentialing providers. The MCO submitted all policy 
regarding disenrollment. All practice issues regarding disenrollment, information to members, and 
policy reviewed met the requirements of this regulation. Missouri Care had approved agreements 
with their subcontractors, and actively worked to ensure that they were complying with necessary 
regulations. The MCO had one remaining policy to be completed and submitted to the SMA for 


approval. 


Measurement and Improvement 

Missouri Care had instituted clinical practice guidelines to ensure consistent service delivery and 
decision-making regarding healthcare for members. These guidelines were disseminated through 
provider training initiatives in the Region. The guidelines were in the Provider Manual and included 


asthma, diabetes, CHF/COPD and ADHD. 


The MCO did operate an internal quality assessment and improvement program., called the Quality 
Management Oversight Committee. (QMOC), consisted of members from each department. Each 
internal department participated in developing quality metrics to assess their performance. The 
large group met quarterly and implemented recommended changes that improved organizational 
performance. The MCO initiated a Service Improvement Committee (SIC) comprised of all 
department managers. This group discussed all member complaints. The head of Medical 
Management then reviewed all medical records associated with these complaints. Ifa problem 
remained unresolved, the record was forwarded to the Chief Medical Officer (CMO). The CMO 
reviewed the record and presented the complaint to the Medical Quality Committee and Quality 
Management Oversights Committee. Any necessary corrective action occurred immediately after 


case review. 


The MCO provided two Performance Improvement Projects for validation. The detailed findings 
are located in the appropriate section of this report. The MCO used the hybrid method to validate 
two Performance Measures and had some difficulty in meeting the requirements for medical records 
submission. Missouri Care did have a health information system (HIS) capable of meeting the MC+ 
Managed Care program requirements. The system did not provide encounter validation data in a 


format that allowed the required analysis. These were areas in need of improvement. 
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Ratings for compliance with Measurement and Improvement regulations (63.6%) reflected need for 
improvement in data submitted and the need for required policy. The MCO continued to have 
MC+ Managed Care Program required policy that was not in place. This policy should be completed 
and submitted to the SMA for their final review and approval to achieve compliance with these 


regulations. 


Grievance Systems 

Ratings for Grievance Systems regulations ((94.4%) reflect that Missouri Care was substantially 
compliant regarding the policy and practice of their grievance and appeal system. Files for both 
grievances and appeals, filed by members and providers, were reviewed on-site. All files followed 
prescribed policy and timelines. Notices to members were sent within required timeframes and 
contained all required information. This information included the member’s ability to file a State Fair 
Hearing simultaneously with an MCO appeal, or later if the outcome of an appeal was not favorable 
to the member. All policy and information to members included the message that members can 
maintain healthcare coverage while an appeal was pending, and explained the member’s 
responsibility to pay for charges if a decision was reached to deny the disputed service. The Medical 
Director was involved in many disputed issues and made every attempt to ensure that members 
obtained the healthcare they needed. When an appeal was filed, it was recorded in the MCO 
tracking system to ensure that another qualified individual, not involved in the original decision, 
reviewed the information submitted to obtain an independent appeal decision. The MCO took the 
grievance and appeal system very seriously and used information generated to inform their quality 


improvement process. 


Missouri Care had not submitted required policy regarding properly informing providers and 
subcontractors about the grievance system. This policy must be completed and submitted to the 


SMA for their final review and approval to be fully compliant with these regulations. 


Summary and Follow-up 

Behavioral Health 

Utilization and member satisfaction were two prominent follow-up issues regarding behavioral 
health services for Missouri Care. The MCO expected to give more attention to these issues during 
the coming year with the change to Missouri Care direct management of behavioral health services. 


Missouri Care staff indicated that since they became more involved in utilization management, 
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tracking services provided and service integration had improved. Many of the specific processes 
remained under development as this was still new to Missouri Care. 

Overall the provider network, including participating hospitals, had increased with the change to 
direct provision of behavioral health services by Missouri Care. Most CommCare providers chose 
to participate in the new Missouri Care network. The MCO believed no member lost services, or 


suffered negative therapeutic consequences from this change. 


The studies that were conducted previously by CommCare were not available, so follow-up could 
not occur. The current Missouri Care Medical Director had convened an ADHD task force 
following a review of pharmacy data. This group included physical and behavioral health providers, 
school representatives, and parents. Rural counties were represented. The task force goal was to 
develop practice guidelines and a tool kit for assessment and service coordination for members and 
providers. The Medical Director hoped that a secondary outcome from the work this group was 


engaged in would be better integration of behavioral and physical health services. 


Electronic Claims Submission 

Missouri Care stated that their previous rate of electronic claims submission had been in the 40-50% 
range. The MCO was using the expertise of their consulting organization, Schaller Anderson, Inc., 
to make improvements in this area. By the end of 2004 their rate of electronic claims submissions 
improved to 55%. Their 2005 goal was 70%. The MCO believed they would meet this target 


because of a change in laboratory subcontractors. 


EPSDT Initiatives 

The rate for EPSDT screenings for children and adolescents continued to be an issue for Missouri 
Care. They started several initiatives to increase this rate. The MCO was sending overdue lists to 
PCPs in addition to the reminder cards sent to members. Providers were asked to send additional 
reminders to their member-patients. Missouri Care was attending Back-to-School fairs and 
completing EPSDT screenings on-site. This initiative led to 105 high school age adolescents 
obtaining screenings in the school districts where this occurred. Adolescents who were screened 


received a pass to the University of Missouri athletic event of their choice as an incentive for having 
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the examination. Missouri Care planned to provide on-site screenings again this fall, and hoped to 
include more districts in the process. Another sample of adolescents who had not received EPSDT 
screenings were offered movie passes for obtaining the examination. Fifty-five members took 


advantage of this incentive. 


Special Health Care Needs 

Missouri Care used the short form of the Child and Adolescent Health Measurement Initiative 
(CAHMI) survey with children identified as requiring an assessment for special health care needs. 
The MCO reported that approximately 10% of all referrals met the threshold for receipt of case 
management services. The MCO believed this tool was effective although they had not conducted 


any formal evaluative study. 


Missouri Care was collaborating with the Boone County Health Department. The Health 
Department provided Prenatal Case Management services for the MCO. They were also 
reimbursed for identification of members in need of case management and completion of risk 


assessments. 


STRENGTHS 


|. Missouri Care convened an ADHD Task Force that was developing practice guidelines. The end 
product was expected to be expanded guidelines to address both behavioral and physical health 
best practices. Missouri Care believed that this process would lead to procedures for improved 
communication between PCPs and other specialty providers. 


2. Missouri Care was working with the SMA to allow the use of Tele-psychiatry services in at least 
three communities where a child psychiatrist was not available. This would create a method to 
increase the availability of a mental health specialty that was a limited resource for the Medicaid 
population (i.e. rural areas). 


3. Beginning in August 2004, Missouri Care began providing behavioral health services directly, 
rather than through a subcontract with a Behavioral Health Organization (BHO). The goal was 
to create a more integrated approach to providing mental health services to Missouri Care 
members. 


4. Missouri Care had a strong partnership with the University Physicians Network, due to their 
relationship with the University of Missouri. As a result, the MCO and its members had access 
to an extensive network of PCPs and specialists in this rural MC+ Region. 


5. There was a strong case management team at Missouri Care. The case managers were 
knowledgeable about community resources and local services. They exhibited a strong 
commitment to member services. 
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AREAS FOR IMPROVEMENT 


Missouri Care staff needed to complete all documentation requirements to ensure compliance 
with the MC+ Medicaid Managed Care contract and the federal regulations. 


2. Missouri Care needed to develop a systematic review process to ensure timely policy updates 
and annual reviews, as required by the MC+ Medicaid Managed Care contract or federal 
regulations. 

3. Staff within Missouri Care understand their individual roles and responsibilities, but were 
sometimes unsure how they influenced other units within the organization. This created a gap 
in organizational knowledge that could lead to incomplete service delivery. 

RECOMMENDATIONS 

|. Continue completion of documentation with the same level of importance observed in the high 
standards reflected in the daily practice within the MCO. The MCO should establish a policy 
and procedure for the annual review of all MCO policies. 

2. Continue MCO development in the area of utilization of available data and member information 
to drive change and create opportunities for organizational growth and development. 

3. Integrate knowledge of all organizational functions throughout Missouri Care to ensure a more 
holistic approach to member services. 

4. Continue tracking electronic claims submissions and make adjustments throughout the year to 
reach targeted goals in this area. 

5. Continue using creative methods to engage the support and interest of school superintendents 
in providing the EPSDT screenings on-site. Utilize MCO data to identify the districts outside of 
Columbia that have large numbers of MC+ Managed Care members. Missouri Care can market 
this plan and past success to these superintendents. Track all efforts and use this data for future 
planning purposes. 

6. Continue current community-based and local health department partnerships. Use this as a 
method of engaging other rural health departments who may have the capacity to provide 
services in underserved areas. 
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FAMILY HEALTH PARTNERS 


The previous sections of the 2004 EQRO report present the purpose and objectives technical 


methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 


Document Review 


Family Health Partners supplied the following documentation for review: 
Lead Performance Improvement Project Description 

Lead Performance Improvement Project Worksheet 

Lead Case Management Referral Rates 

Emergency Room Utilization Performance Improvement Project Description 
Nurse Advice Line Utilization chart 

Emergency Room Utilization Chart 


Interviews 

Project Leaders, participants, and the Medical Director were interviewed on-site by the EQRO 
Project Director on Monday, March 14, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided as were references for logic 
model development and health services research methods. Specific questions regarding each study 
were also addressed: 


Lead Performance Improvement Project 
e Why was the target population restricted to those continuously enrolled since birth? 


Why was the target population restricted to those with no lead claims since birth? 

What is the difference between the lead information letter and the educational letters? 

To what group is the reminder call/letter group(s) being compared? What are the age groups? 

What was the rate of BLL in the target population prior to implementation of the intervention(s)? This is 

the baseline (pre-intervention) rate. 

e The hypotheses suggest well visit, lead toxicity rate, sick child visits, and hospitalization for chelation therapy 
as outcomes. 

e How is the rate of BLL testing per member determined? 

e When were the interventions put into place? 

e How often are measures examined? 
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Emergency Room Utilization Performance Improvement Project 
e Was there a distinction made between non-emergent and emergent emergency room use made? 


e Were there measures other than emergency room utilization that examined quality of care, access, or 
health-related outcomes? 


FINDINGS 

The first PIP evaluated was “Emergency Room Utilization.” Family Health Partners identified that 
90% of members who had emergency room visits more than twice in 60 days had not been seen by 
their physicians. Additionally, emergency room utilization increased by |7% and 43% for children 
and adults, respectively. The study questions were clear and understandable and imply some 
process measures to be used. They were: |) “Do mailings of educational materials through direct 
mail and PCP offices increase the use of Nurse Advice Line services”?; and 2) “Do letters and 
phone calls to members who are utilizing the ER for non-emergency services affect and decrease 
overall ER use in the entire population”? The rationale presented was clear and identified a 
problem. This project was focused mainly on emergency room utilization. More emphasis should 
be placed on the improvement of care. For example, the study could concentrate more on the 
reasons why a significant number (90%) of their members who had ER visits had not had a PCP visit. 
Although this issue was mentioned in the description of the intervention, it did not receive attention 
in the study. The focus on the adult dental service education implies some knowledge of a possible 
health outcome to be incorporated in the study design. The intervention focused on educating 


members about the availability of primary care services and a Nurse Advice Line. 


Interventions that were targeted toward identification of trends and reasons for inappropriate ER 
utilization (e.g., well-care utilization), could also be used to determine the effectiveness of the 
intervention. The rate of Nurse Advice Line calls and the rate of emergency room utilization 
measures were objective and clearly defined. It was not clear how many members were included at 
each of the baseline measurement points, or how many members for which time period constituted 
the proportions (e.g., 72% of calls to the Nurse Advice Line). There were no long-term outcomes 
stated. The introduction indicated a relation between the coordination of care, well-care, and 


quality of care. Some of these could have been measured. 


The description of the study population should have been more thorough, to be clear about 
whether the study findings relate to MC+ Members only or other possible member groups, and 
whether all categories of MC+ Members were included in the study population. It was reported 


that all currently enrolled members were included in the study. However, no information on age, 
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gender, or race was provided. Age ranges of child and adult populations should be described. The 
study did appear to capture those attending higher-volume facilities for unstated reasons. This 


should be more clearly described. 


There was no sampling conducted. The data collection plan was clearly described. The method for 
collecting claims data could be presented in more detail. The data analysis plan referred to both 
pre- and post- intervention comparisons and statistical quality control charts, but no analyses were 
presented. Separate charts for adult and pediatric populations are recommended. There was 
reference to a control group but no rationale or data analytic plan presented. Data on the 
effectiveness of the implementation should also be collected to identify areas for fine-tuning the 
intervention (e.g., the number of members attempted to contact and number actually contacted). 
No specific barriers to the intervention were presented. The process developed for calling 


members who use the ER for non-emergency services was not described. 


The charts presented in the study are a good start for data presentation. Upper and lower 
confidence levels should be included. The number of members for each measurement point as well 
as numerators and denominators for proportions should be included in any data presentation. It is 
recommended that 95% confidence interval should be employed for assessing change over these 
indices across time. The interpretation of the extent of success of the intervention was not 
provided. There was a quarterly update and quarterly data collection beginning in January 2004. 
The study findings were submitted November 2004 yet no pre- or post-intervention comparisons 
were presented. The description of the measures provided a clear understanding of the sources of 
data, the method of data collection, the participants included, and the tools used at each 
measurement point. In terms of documented improvement in process or outcomes of care, there 
was no interpretation of findings presented. The questions that remained included: |) “Was there 
statistically significant increase in nurse advice line calls and a decrease in ER utilization?”; 2) “What 
misuses of the emergency room were identified?”; 3) “What root causes were identified?” A 


description of how the intervention was related to the measure improvements should be included. 


This PIP is not likely to provide a credible evaluation of the impact of member education on the 
process of care without substantial revision. The study focused primarily on utilization and not 
enough on improvement of processes of care and therefore does not qualify as a Performance 


Improvement Project. There were some implications in the introduction that the use of emergency 
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rooms was related to quality of care, coordination, and well-care, however there was no 
measurement of these concepts. The data analysis plan was vague and did not appear to have been 


applied. 


Family Health Partners should consider a shift in the study’s focus from utilization to identification of 
causes for inappropriate utilization. Member and provider surveys can help in identifying the source 
of the problem. Another intervention that was implied in the discussion was access to dental care 
for adults. This could be another area for measurement of the effectiveness of the intervention. 
Identify measures related to health care access or outcomes such as the use of well-care as the 
result of the intervention. Use available data to identify the target or point at which success will be 
considered to be attained (baseline data may be helpful in identifying a target for 10% or 15% 


improvement). 


The topic of the second PIP evaluated was, “Improvement of Lead Screening rates among members 
between the ages of 6 months and 3 years”. The current rates of lead screening among these 
groups should have been presented as a rationale for targeting this topic as a problem for 
improvement. The study questions were: |) “Do letters and reminder calls to children identified as 
needing blood lead testing result in increased blood testing for those children?”; and 2) “Do letters 
and reminder calls to children identified as needing blood lead testing result in increased referrals to 
case management for high lead levels?”. The study questions were stated clearly in writing, but will 
likely result in difficulty assessing the effectiveness of the intervention. It is recommended that this 
be redesigned to include all of the interventions and all children identified as eligible. The 
identification of children with no blood lead testing should be considered part of the intervention 
and the rates of all children eligible for the intervention and study should be assessed prior to and 
following implementation of the intervention. This could be a design where the same population is 
followed (comparing those who were able to contacted with those who were not on blood lead 
level rates) over time; or where statistical quality control charts for the rates of all eligibles at 
different points in time are examined to determine the direction of the trend and the statistical 


significance of the change. 


The intervention involved identification and outreach for members who had not received lead 
screening, education of providers and members on lead testing, and education of health departments 


regarding coding and billing for lead screening. 
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This was a nice multi-pronged intervention, targeting different systems involved in lead screening, for 
addressing low lead screening rates. It included agencies, members, and providers. The quality 
improvement initiative and intervention appeared to include those from 6 months to 3 years in age 
who were continuously enrolled during an unspecified time period, living in Johnson County and 
attending a participating agency. Study questions and hypotheses should be directed toward this 
population. The number of providers, agencies, and members included as well as their 
characteristics should be described. The intervention included a broad range of enrollees, but the 
study was overly restrictive in studying only those who had no lead screening. Also, the rationale 
for including only continuously enrolled members should be described, especially since the target 


geographic region, is considered high-risk . 


The measures and their calculation were not well defined. The study used the rate of blood lead 
testing per member; the rate of case management cases referred for blood lead level greater than 
|Om/dL. The rate of blood lead testing per member is an unusual indicator. The rates would be 
from 0 (no blood tests in a year) to a relatively small number (possibly up to 3 or 4 lead tests per 
member in a year), resulting in a restricted range on the outcome indicator. It is unclear whether 
this was a rate for a particular year or timeframe. The rate for the entire study population should 
be examined, rather than the rate for each member. A simple percent of members tested of those 
who were eligible would be useful as an indicator and would be relevant to public policy and 
managed care policy on the requirements for blood lead testing. The indicators measured were 
different from those implied in the hypotheses. It was unclear how they related to those in the 
hypotheses and why some were measured and others were not. Numerators and denominators for 
rates should be clearly defined. For example, the rate for 3 months to 6-year-olds who were 
continuously enrolled during the specified time period and who received or did not receive phone 
calls could be examined. There were no comparative measures presented as benchmarks. It might 
be helpful to know what the rate of blood lead testing in other high-risk areas of the state or nation 
for comparison purposes. The indicators certainly measure changes in health status and outcomes, 
as the rates of blood lead screening and long-term health outcomes are closely interrelated. 

lt was unclear whether all MC+ Members were included in the intervention (e.g., | 115 Waiver, 
I915b Waiver, children in State custody, consent decree children); or whether one group was 
followed over time, whether all other groups were included, or whether the study applied to MC+ 
Members alone. Measures of the entire population for which the study and intervention applied 


should be employed. Those who were not continuously enrolled were excluded from the analysis. 
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This should be explained. Two measures were identified, but no methods of calculation were 
presented. The sources of data and methods of calculation including formulas should be presented. 
One source of data identified was a case management database. Measures were collected ona 
quarterly basis, but did not apply to the entire population being studied. They were restricted to 
only members who could be contacted by telephone, and those with blood lead levels meeting case 
management criteria. This is overly restrictive and will result in very low numbers as well as 
information that cannot be very well interpreted to assess the impact of the intervention being 
studied. There was a plan to generate a quarterly update report and another reference to semi- 
annual data collection, so it is not clear how often re-measurement points are planned. Given that 
the study was underway, and there was no re-measurement at the time of evaluation, the 


identification of improvement or sustained improvement was not able to be made. 


The study as described and presented has low potential for producing valid findings from which to 
determine whether or not the intervention is effective. The strengths of the study include a multi- 
pronged intervention strategy which could be studied as a whole or in part (identification of 
members in need of blood lead testing, phone calls to members). Reasonable interventions were 
selected and they have a good potential to improve healthcare processes. There were also some 
system-level interventions taking place. With revisions to the data collection and analysis plan, this 
study could produce credible findings to assess the impact of the intervention. The baseline should 
consist of the rate of blood lead levels among all members of the eligible population prior to the 
intervention, with re-measurement at various points during the intervention. Alternatively, a 
comparison group of blood lead level rates for those members who are unable to be contacted 


could provide a method for measuring the impact of contacting and educating members. 


lt is recommended that more attention be devoted to the identification of barriers and development 
of interventions to address those barriers. Identify upper and lower confidence intervals and target 
numbers for identifying success of the intervention. Since the interventions are stratified, identify 
the rationale for so doing and stratify the data analysis by the type of members if there are enough 
eligible members. Rather than restricting measurement to whether or not a lead test was 


conducted, examine the rates of lead testing in each of the populations (for example those 6 to |2 
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months old continuously enrolled since birth) pre- and post-intervention. This would allow for 


statistical significance testing of the strength of the intervention. Describe the interventions and 


how they are calculated. The rate of blood lead levels for |2- and 24-month-olds for those in the 


study population would also be good indicators. 


STRENGTHS 


The study topics were excellent. 


Staff was qualified to plan, implement, and interpret the results of the intervention. There was a 
process of including clinical and health services staff in the planning, implementation, and 
refinement of performance improvement activities and projects. 


There was good use of data as feedback for continued improvement processes. The use of 
Nurse Advice Line data was instrumental in deciding to simultaneously implement an 
intervention of placing a case manager in the emergency room at a local hospital as a pilot 
project. This has been implemented as a full intervention and improvement project at one 
hospital, and can be evaluated in the next PIP. 


Data are planned to be collected and analyzed on a quarterly basis. At least quarterly data 
collection and analysis is strongly supported, to allow for refinements in the intervention(s). 


AREAS FOR IMPROVEMENT 


The findings of the performance improvement project should be completely described and self- 
contained and able to be repeated based on the level of description of the study elements, 
analyses, findings, interpretations, and recommendations. 


2. Process measures of the effectiveness of the intervention at reaching the target population 
should be collected and examined at least quarterly. Measures which examine the effectiveness 
of the process of the intervention are critical in determining whether the intervention was 
implemented as planned. 

3. Trend backward at least one year prior to the intervention for measures using claims data when 
constructing statistical quality control charts. 

RECOMMENDATIONS 

|. For the lead study, use process measures such as the number of members eligible for the study; 
the number able to be contacted by each method (e.g., mail and telephone) for each risk group 
(e.g., low and high) within each cohort (a new one selected every 3 months). 

2. For the lead study, ensure that the majority of the population is included in the definition. For 
example, the 7,455 members from birth to 36 months of age continuously enrolled since birth 
should capture the majority of the population of members from birth to 36 months of age. 
Ensure that the continuous enrollment criteria do not unnecessarily exclude members that may 
be able to benefit from the intervention. 
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Outcome measures for the lead study that should be examined quarterly within cohorts overt 
time include lead toxicity, case management referrals, and blood lead level testing. Over time, 


additional measures implied in the study could be added (e.g., well care visits, sick visits, 
chelation therapy). 


When examining the utilization of any service, establish the link between the use of services and 
some short- or long-term outcomes. For example, access to primary care may serve as a short- 
term proxy measure of the effectiveness of reducing unnecessary emergency department use. 

This may also be associated with other measures of access, quality, or timeliness of care. Ensure 


that studies are not limited to utilization or cost alone. 


When submitting studies for review and interpretation, submit and compile all data in writing, to 
include the entire process of problem identification, study rationale, baseline data, logic model 


and data analysis plan, findings to date, and the interpretation of the findings as well as 
recommendations for future interventions/quality improvement activities, and analysis. 


Validation Of Performance Measures 


METHODS 


This section describes the documents, data, and persons interviewed for the Validation of 


Performance Measures for Family Health Partners. Family Health Partners submitted the requested 


documents on November 23rd, 2004. The EQRO reviewed documentation between November 


23rd, 2004 and February 28th, 2005. On-site review time was used to conduct follow-up questions 


and provide feedback and recommendations regarding the performance measure rate calculation. 


Document Review 


The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by Family Health Partners to the SMA 


e The Baseline Assessment Tool (BAT) submitted by Family Health Partners for the HEDIS 2004 data 


reporting year 
e Qualis Health’s NCQA HEDIS Compliance Audit Report for HEDIS 2004 


e Family Health Partners’ information systems (IS) Policies and Procedures pertaining to HEDIS 2004 rate 


calculation 
e Family Health Partners’ information services (IS) policies on disaster recovery 


e Family Health Partners’ HEDIS 2004 implementation work plan and HEDIS committee agendas for 2004 


e Family Health Partners’ HEDIS 2004 Training Manual for the medical record review process 


e Documentation, data files and source code of the in-house application for immunization rate calculation 


e System edits for the claims management system 


e HEDIS 2004 report numbers for Adolescent Well-Care Visits and Use of Appropriate Medications for 


People with Asthma. 


Family Health Partners (Family Health Partners) submitted the requested documents on November 


23rd, 2004. The EQRO reviewed documentation between November 24th, 2004 and February 


28th, 2005. 
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The following are the data files submitted by Family Health Partners for review by the EQRO: 


e Denominator File Layout.xls 

e HEDIS Denominators.mdb 

e HQMDNP_AIDENOM04.txt 
e HQMDNP_AIDENOM04.xls 
e hqmdnp_astlO_1I7.txt 

e hqmdnp_astl0_17.XLS 

e hqmdnp_astl8_56.txt 

e hqmdnp_ast!l8 56.XLS 

e hqmdnp_ast5_9.txt 

e hqmdnp_ast5_9.XLS 

e hqmdnp_WC3_04.txt 

e hqmdnp_WC3_04.XLS 

e 2004 Mosaic DL.mdb 

e 2004 MOSAIC DL.txt 

e 2004 Mosaic DLI.mdb 

e Adolescent Immunization 2003 Medical Record Review.mdb 
e Adolescent Immunization 2003 Medical Record Review.txt 
e Adolescent Immunization 2003 Medical Record Review.xls 
e = Al_lImmunizationHistory.xls 

e —_ File Layouts.xls 

e = tblIAl_ ImmunizationHistory.txt 
e HQMDOP_AI2004HYBR.txt 
e HQMDOP_AI2004HYBR.xls 
e HQMDOP_AST10-17.txt 

e HQMDOP_AST10-17.xls 

e HQMDOP_AST18-56.txt 

e HQMDOP_AST18-56.xls 

e HQMDOP_AST5-9.txt 

e HQMDOP_AST5-9.xls 

e HQMDOP_WC3_04.txt 

e HQMDOP_WC3_04.xls 

e Numerator File Layout.xls 


Interviews 

The EQRO conducted on-site interviews with Janet Benson, Bob Clark, and Jenny Hainey at the 
Family Health Partners in Kansas City on Monday, March |4th, 2005. This group was responsible 
for calculating the HEDIS performance measures. The objective of the visit was to verify the data, 


methods and processes behind the calculation of the three HEDIS 2004 performance measures. 


FINDINGS 

Family Health Partners used the Administrative Method for calculation of Adolescent Well-Care 
Visits and Use of Appropriate Medication for People with Asthma measures. The Hybrid Method 
was used for the Adolescent Immunization Status measure. MCO to MCO comparisons of the 


rates of Adolescent Immunization Status Combination #1, Adolescent Well-Care Visit, and Use of 
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Appropriate Medication for People with Asthma measures were conducted using two-tailed z-tests. 
For comparisons that were statistically significant at the 95% confidence interval (Cl), the z-score 


(z), the upper and lower confidence intervals (Cl), and the significance levels (p < .05) are reported. 


The rate for the HEDIS 2004 Adolescent Immunization Status, Combination #| reported to the 
SMA and the State Public Health Agency (SPHA) by Family Health Partners was 58.88%. This was 
significantly higher than the statewide rate for MC+ MCOs (14.36%; z = 1.48; 95% Cl: 44.02%, 
73.74%; p > .95). 


The rate for Family Health Partners for the HEDIS 2004 Adolescent Well-Care Visit measure was 

32.93%, which was comparable to the statewide rate for all MC+ MCOs (30.13%; z = .47, 95% Cr: 

28.05%, 37.81%; n.s.). The rate for Family Health Partners for the 2004 HEDIS Use of Appropriate 
Medications for People with Asthma was 66.57%, which was significantly higher than the statewide 

rate for MC+ MCOs (63.92%, z = .77; 95% Cl: 62.93%, 70.21%; p > .95). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

The information systems management policies and procedures for rate calculation were evaluated 
consistent with the Validating Performance Measures Protocol. This included both manual and 
automatic processes of information collection, storing, analyzing and reporting. The EQRO was 
provided with a demonstration of the in-house application developed for calculation of the 


immunization measure. This used an MS Access data repository for retrieval and analysis. 


For all three measures, Family Health Partners was found to meet all criteria for producing complete 
and accurate data (see Attachment V: Data Integration and Control Findings). There were no 
biases or errors found in the manner in which Family Health Partners transferred data into the 
repository used for calculating the HEDIS 2004 measures. Family Health Partners used an external 


vendor application module for rate calculation. This module was part of the claims management 
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system, MC 400, a product of OAO HealthCare Solutions, Inc. This module was NCQA-certified 
for the 2002 HEDIS rate calculation, but the vendor didn’t seek recertification for the 2004 
reporting year. The EQRO was provided with a demonstration of the HEDIS reporting module of 
MC400 along with the data flow and integration mechanisms for external databases for these 


measures. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures appeared adequate, however, the 
processes were not well-documented. (see Attachment VII: Data and Processes Used to Calculate 
and Report Performance Measures). Family Health Partners met nearly all criteria that applied for 
all three measures. The two criteria that were not met involved the use of statistical significance 
testing to document changes in performance over time and using this information to support claims 
of improvement or stability in performance over time. Although many auditors conduct statistical 
tests to examine various aspects of data validity, such tests should be used by Family Health Partners 
to assess the significance of change related to quality improvement activities and operational 
changes. One creative aspect of data validation implemented by Family Health Partners involved the 
calculation of the Adolescent Immunization Status measures using administrative data and relaxing 
the time parameters employed by HEDIS Technical Specifications. A comparison of the rates of 
immunization status as documented through HEDIS measures and through examination of whether 
an immunization occurred at all were increasingly similar over the past several years. This suggests 


more timely administration of immunizations. 


Processes Used to Produce Denominators 

Family Health Partners met all criteria for the processes employed to produce the denominators of 
all three performance measures (see Attachment X: Denominator Validation Findings). This 
involves the selection of eligible members for the services being measured. A sample of 

4| |lmembers were reported and validated for the Adolescent Immunization Status, Combination #| 
measure. A total of 10,188 eligible members were reported and validated for the Adolescent Well- 
Care Visits measure; and 2,429 eligible members were reported and validated for the denominator 
of the Use of Appropriate Medication for People with Asthma measure. Age ranges, dates of 
enrollment, medical events, and continuous enrollment were programmed to include only those 


members who met HEDIS 2004 criteria. Denominators in the final data files were consistent with 
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those reported on the DST for all three measures. All members were unique and the date of birth 


ranges were valid. 


Processes Used to Produce Numerators 
All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2004 criteria (see 


Attachment XIll: Numerator Validation Findings). 


For the HEDIS 2004 Adolescent Immunization Status, Combination #| measure, Family Health 
Partners appropriately included administrative events from the State Public Health Immunization 
Registry (MOHSAIC). Review of the administrative hits validated |30 of |34 hits in the files 
provided for review by the EQRO. Twenty-nine (29) of 30 medical records requested for review 
were received, and 27 records resulted in validated hybrid hits. As a result, the medical record 
review validated 97 of the 108 hybrid hits reported. The estimated bias from medical record review 
was 3.65%. Based on the number of hits validated by the EQRO, the rate calculated by the EQRO 
was 55.23%. The total estimated bias for the Adolescent Immunization Status, Combination # | 


measure was 2.63% overestimate of the rate. 


Family Health Partners used Administrative Method to calculate HEDIS 2004 Adolescent Well-Care 
Visits measure. There were a total of 3,355 numerators reported before exclusions and 3,355 
reported in the numerator of the measure. It is unclear what exclusions were made, as there are 
none permitted in the HEDIS Technical Specifications for either Hybrid or Administrative Method 
calculation of this measure. The dates of service were 100% valid. The visits were identified using 
CPT codes 99383-99385, 99393-99395 and ICD-9CM codes V20.2, V70.0, V70.3, V70.5-6, and 
V70.8-9. There were 308 members with codes that did not meet the criteria provided in the 
NCQA Technical Specifications for Adolescent Well Care Visits. Some of these were CPT codes 
59400, 59510, 99222, 99223, 99244, 99245, 99254, 99255, 99381, 99384, 99385 and 99392. Most of 
these were codes for general and specific obstetric visits, while some represented inpatient or infant 
exams. The HEDIS 2004 Technical Specifications allow for counting visits that are not for purposes 
other than well-care (p. 218): “Preventive services may be rendered on the occasion of visits other 
than well-care visits. If the specified codes are present, these services count, regardless of the 


primary intent of the visit.” However, the codes included are not specified in the Technical 
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Specifications. The rate calculated by the EQRO was 29.91%. The estimated bias from the 


incorrectly included CPT codes was a 3.02% overestimate of the rate. 


The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was the third 
measure validated. There were a total of 1,617 numerators validated. Eleven different diagnosis 
codes were provided and of those, only ten matched with NCQA specified codes for this measure. 
However, all events were associated with prescription of a formulary medication for Asthma. The 
dates of birth and dates of service were in the correct range and valid. The dates of enrollment 
were not provided. The NDC code range was valid. The rate calculated by the EQRO was 66.57%, 


with no bias in the reporting of the measure. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Adolescent Immunization Status, Combination #| measure. 
Attachment XII; Impact of Medical Record Review Findings and Attachment XV: Sampling Validation 
Findings were completed for each of these measures. Family Health Partners was compliant with all 


specifications for sampling processes. 


Submission of Measures to the State 

Family Health Partners submitted the DST for each of the three measures validated to the SPHA 
(the Missouri Department of Health and Senior Services) in accordance with the Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 


Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

The following tables summarize the estimated bias in reporting each of the measures and the final 
validation findings. Table | shows small overestimates of the Adolescent Immunization Status, 
Combination #1 and Adolescent Well-Care Visits measures, with no bias observed in the calculation 


of the Use of Appropriate Medications for People with Asthma measure. 


Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures 


Measure Estimate of Total Bias | Direction of Estimate 
Adolescent |mmunization Status, Combination #1 2.63% Overestimate 


Adolescent Well-Care Visits 3.02% Overestimate 
Use of Appropriate Medication for People with Asthma 0.00% None 
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Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 
measure. Table 2 shows the final audit findings for each measure. The Adolescent Immunization 
Status and Adolescent Well-Care Visits measures were Substantially Compliant, as there was no 
significant bias associated with the overestimated rates. The Use of Appropriate Medications for 


People with Asthma measure was Fully Compliant with State specifications. 


Table 2. Final Audit Rating for HEDIS 2004 Performance Measures 


Measure 
Adolescent Immunization Status, Combination #1 Substantially Compliant 


Adolescent Well-Care Visits Substantially Compliant 

Use of Appropriate Medication for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by the MCO. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Family Health Partners effectively integrates the MOHSAIC data into administrative rates for 
calculation of immunization rates. The sample files are sent to the SPHA for identifying 
members with an immunization history. The SHPA returns a file with relevant immunization 
information for Family Health Partners’ members. MOHSAIC data comprises a majority of hits 
within the administrative hits. 


2. Rates for the HEDIS 2004 Adolescent Immunization Status, Combination #I and Adolescent 
Well-Care Visits measures were higher than the National Commercial averages for these 
measures. 


3. Therate for HEDIS 2004 Adolescent Immunization Status, Combination #1 measure was also 
higher than the National Medicaid average for the measure. 


4. There was a well-designed HEDIS reporting module of the MC400 system. This module was 
certified by NCQA in 2002, but was not certified for 2004. Audit checks were conducted on 
this system by NCQA-certified auditors on an annual basis to confirm the system’s output. The 
system generates appropriate production logs to track errors. 


5. Data from external sources such as the pharmacy vendor AdvancePCS and data from medical 
record review was efficiently integrated. The format of data mapped in the integration process 
is automatically validated and also manually checked. 


6. There were effective validity checks through automated system edits. The claims system has 
internal edit checks for every transaction. It also has an error handling mechanism that validates 
the process. 


7. Family Health Partners had a separate application for combining the immunization data from 
medical record reviews and administrative search. An MS Access database acts as a data 
repository for the application. This integrates the data from the claims system and effectively 
combines it with external data to produce the final immunization rates. This application was 
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well-documented and produces accurate results. The data fields were clearly defined and 
business rules are well-documented. 


AREAS FOR IMPROVEMENT 


Family Health Partners programming staff did not have detailed knowledge of the HEDIS 2004 
rate calculation process due to employee turnover and lack of documentation. 


There is a need to develop HEDIS measure calculation policies and integrate them with the 
information systems policies, as they are closely intertwined. Family Health Partners is in the 
process of documenting policies and providing an effective knowledge transfer mechanism. 


Data analysis should incorporate tests of statistical significance to assess whether the observed 
changes in rates are related to a specific intervention. 


Invalid event codes were used to include members in the numerator for the HEDIS 2004 
Adolescent Well-Care Visits measure, and exclusions appear to have been made when not 
allowed. 


RECOMMENDATIONS 


Family Health Partners should improve documentation of the HEDIS rate calculation process. 
Document processes in such a manner that subsequent employees can calculate and process the 
measures consistent with the established procedures. 


Integrate HEDIS policies into the information systems policies, as data are highly dependent on 
reliable and responsive information systems. 


Conduct data validation checks on all parameters associated with each of the measures to 
ensure that the HEDIS 2004 Technical Specifications were followed, especially with regard to 
Adolescent Well-Care Visits numerators and allowable exclusions. 


Conduct and document statistical comparisons on rates from year to year. 


lt is recommended that Family Health Partners staff involved in the coordination of calculation 
of performance measures receive training in the procedures and parameters for each measure. 
NCQA conducts training for the calculation of HEDIS measures. System edits should be 
incorporated to ensure the use of valid codes in identifying numerators. 


Validation of Encounter Data 


FINDINGS 


The findings for the encounter data validation are organized according to the encounter data 


evaluation questions presented in the Technical Methods section for the encounter data validation in 


the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 
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What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical fields? 


For the Medical claim type, there were a total of 107,089 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Outpatient Recipient ID field was |00.00% complete, accurate and valid. 


3. The Outpatient First Date of Service field was 100.00% complete and accurate, and 100.00% 
valid (with rounding). Two invalid dates of service ranged from 01/05/2001 — 12/16/2003. 


4. The Outpatient Last Date of Service field was 100.00% complete and accurate, and valid. 
5. The Outpatient Units of Service field was 100.00% complete accurate and valid. 


6. The Outpatient Procedure Code field was |00.00% complete and accurate, with 99.94% 


containing valid values. There were I9 invalid procedure codes of “Y0025”, 43 invalid entries of 
“Y0029”, and two invalid entries of “Z0020”. 


7. The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 43.78% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 14.85% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
7.31% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00 % complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Dental claim type, there were a total of | 7,123 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All of the fields examined were |00.00% complete, 


accurate and valid. 


For the Home Health claim type, there were a total of 242 encounter claims paid by the SMA for 
the period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 
accurate, and valid except for the second through fifth Diagnosis Code fields. The second, third, 
fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for completeness, 
accuracy, and validity established by the SMA (51.65%, 20.66%, 17.36%, and 0.00%, respectively). 


The remaining fields were blank (incomplete, inaccurate, and invalid). 
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For the Inpatient claim type, there were a total of 13,769 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 


The Admission Type field was 100.00% complete, accurate and valid. 


= mw NM 


The Admission Date field was |00.00% complete and accurate; and 96.56% valid. There were 
473 invalid dates ranging from | 1/16/2003 — 12/31/2003. 


7 


The Discharge Date field was 100.00% complete and accurate; and 96.05% valid. There were 
544 invalid dates ranging from 04/01/2004 — 04/26/2004. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


2 eS Ss SS 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (89.50, 68.68%, 35.46%, and 4.13%, 
respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 95.56% valid. There 
were 544 invalid dates of service ranging from | 1/16/2003 — 12/31/2003. 


l1. The Last Date of Service field was 100.00% complete and accurate, and 96.05% valid. There 
were 544 invalid dates of service ranging from 04/01/2004 — 04/26/2004. 


12. The Revenue Code field was 99.99% complete, accurate, and valid. The one invalid field 
contained an entry of “080”. 


13. The Units of Service field was |00.00% complete and accurate, and 99.97% valid. There were 
four invalid values of “‘O0000”’. 


For the Outpatient Hospital claim type, there were a total of 82,813 encounter claims paid by the 


SMA for the period January |, 2004 through March 31, 2004. 


|. The Outpatient Hospital Claim Type field was 100.00% complete, accurate and valid. 


2. The Recipient ID field was 100.00% complete, accurate and valid. 
3. The First Date of Service field was 100.00% complete and accurate, and valid. 
4. The Last Date of Service field was 100.00% complete and accurate, and valid. 
5. The Units of Service field was 100.00% complete, accurate and valid. 
Performance Management Solutions Group 298 
a Division of Behavioral Health Concepts, Inc. 
2716 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouR!I MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


6. The Outpatient Procedure Code field was 100.00% complete and accurate, and 95.83% valid. 
This field requires five alphanumeric characters. The following are the 3,455 invalid entries 
found. 





Frequency Code 
| 0250 

| 0721 

| 9970 
29 99999 
| W0037 
5 W 1363 
69 W 1365 
9| X4003 
52 X4006 
3 X4009 
30 X4010 
742 X401 | 
6 X4014 
4 X4015 
77 Y3114 
8 Y3115 
2 Y3116 
5 Y3117 
2,261 Y7506 
55 Y7507 
2 Y7508 
10 Y7509 


The Outpatient Hospital Revenue Code field was 100.00% complete, and 99.95% accurate and valid. 


This field required three digits, ranging from “100” — “999”. The 43 invalid codes were as follows: 





Frequency Code 
37 0 
| 00 
| 007 
| 01 
2 030 
| 035 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 
Although the second through fifth Diagnosis Code fields are optional according to the Health Plan 


Record Layout Manual, they fell well below the 100% threshold established by the SMA for this 
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validation. The second Diagnosis Code field was 55.04% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field was 
26.61% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 13.40% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The fifth 
Diagnosis Code field was .95% complete, accurate and valid. The remaining fields were blank 


(incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were a total of 80,722 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, accurate and 


valid data for all fields examined. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Family Health Partners, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. Dental, 
Home Health, and Pharmacy claim type fields examined were |00.00% complete, accurate, and valid 
(see previous findings). The Outpatient Procedure Code field in the Medical claim type contained 
invalid procedure codes; while the Revenue Code and Units of Service fields for the Inpatient claim 
type contained some invalid data. For the Hospital Outpatient claim type, there were invalid codes 


for the Outpatient Procedure and Revenue Code fields. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the rates of Inpatient, 
Medical, Pharmacy, and Home Health claim types were consistent with the average for all MC+ 
MCOs, while the rates for Dental and Outpatient Hospital claim types were significantly higher than 
the average for all MC+ MCOs. This suggests that the data are complete and that there is better 


utilization of dental and preventive services among Family Health Partners members. 


The findings from the performance measure analysis show significantly higher rates of administrative 


hits for the HEDIS 2004 Adolescent Immunization Status, Combination #1, Adolescent Well-Care 
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Visits, and Use of Appropriate Medications for People with Asthma measures. Family Health 
Partners also identified a significantly higher rate of eligible members for the Use of Appropriate 
Medications for People with Asthma, which may indicate either more members meeting the HEDIS 


criteria for asthma, or more complete claims data. 


To What Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 
the period of January |, 2004 through March 31, 2004 for medical record review. Of the 95,566 
Medical encounter claim types in the SMA extract file for January |, 2004 through March 31, 2004, a 
total of 100 encounters were randomly selected. This was an oversample, as the minimum required 
sample size was 88 per MC+ MCO. Providers were requested to submit both medical records and 
claim forms or claim histories for review. There were /| medical records (71.0%) and 18 claim 
forms (18.0%) submitted for review. Encounters for which no documentation was submitted were 
unable to be validated. The match rate for procedures was 50.0%, with a fault rate of 50.0%. The 


match rate for diagnoses was 45%, with a fault rate of 55.0%. 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file was missing information (n = 97) 
with no incorrect information. For the diagnosis description in the medical record, the reasons for 
diagnoses not matching the SMA extract file were missing or illegible information (n = 53), or no 
match with the description of the symptoms based on the information in the medical record (n = 8). 
For the diagnosis code on the claim form, the reasons for diagnosis codes not matching the SMA 
extract file were missing or illegible information (n = 88), or possible data entry or recording error 
by the provider (n = |). The reason for the diagnosis descriptor on the claim form not matching the 


SMA extract file included missing or illegible information (n = 66). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 


file not being supported by documentation in the medical record were missing information (n = 10), 
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downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 4), incorrect codes (n = 6), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = 4), illegible information (n = |), and not enough information to code (n=1). For the 
procedure code on the claim form, the reasons for procedure codes not matching the SMA extract 
file were missing or illegible information (n = 4), or incorrect codes (n = |). For the procedure 
description on the claim form, the reason for procedure codes not matching the SMA extract file 
was missing or illegible information (n = 85). Of all the procedure and diagnosis codes, there were 


none that were incorrect. The main reason for the fault rates was missing or illegible information. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, Family Health Partners was able to submit files in readable formats, but 
in several cases, claims data did not appear complete, as some records appeared shorter than 
expected for the file layout. The following section summarizes the issues found when attempting to 


load files submitted for encounter data validation analysis. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


A letter from Jenny Hainey, dated December 29, 2004 indicated “‘Family Health Partners HCFA Paid 
encounters, Family Health Partners UB92 paid encounters, Transportation HCFA paid encounters, 
Pharmacy paid encounters, behavioral health paid encounter, dental paid encounters, Family Health 
Partners HCFAs Unpaid encounters, and Family Health Partners UB92s unpaid encounters.” There 
was no indication of which files correspond to these descriptions, and no other documentation 


regarding file layouts. 


Performance Management Solutions Group 302 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


I. BPHPaidEncounters.txt 27KB Text Document 12/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. 


2. CCHPaidEncounters.txt 28KB Text Document [2/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. There were incomplete (short) 


records and an incorrect address format. 


3. FHHPaidEncounters.txt 809KB Text Document [2/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. 


4. FHUPaidEncounters.txt 309KB Text Document 12/29/2004 4:02 PM 


This file was in UB-92 format and was able to be loaded for analysis, but the records were short, and incomplete 


5. MTHPaidEncounters.txt 41KB Text Document [2/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. 


6. PCSPaidEncounters.txt 213KB Text Document [2/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. 


A total of 273 records were able to be loaded for analysis. 


7. FHHNotPaidEncounters.txt 26/7KB Text Document 12/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis 


8. FHUNotPaidEncounters.txt !08KB Text Document 12/29/2004 4:02 PM 
This file was in NSF/CMS 1500 format and was able to be loaded for analysis. 


STRENGTHS 


|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 


2. The critical fields evaluated for the Dental, Home Health and Pharmacy claim types were 
|00.00% complete, accurate, and valid. 


3. Diagnosis and procedure code fields for all claim types were complete, accurate, and valid. 


4. The match rate between the medical record and SMA encounter claims data was comparable to 
the average for all MC+ MCOs for the procedure code and description. 


AREAS FOR IMPROVEMENT 
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|. The Outpatient Procedure Code field for the Medical claim type contained invalid codes. 


2. The Revenue Code and Unit of Service fields for the Inpatient claim type contained invalid 
codes. 


3. The Outpatient Procedure and Revenue Code fields in the Outpatient Hospital claim type 
contained invalid codes and was 95.83% valid. 


4. The match rate between the medical record and SMA encounter claims data was significantly 
below the average for all MC+ MCOs for the diagnosis code and description. 


RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the NSF/CMS 
1500 file layout for the Outpatient Procedure Code and run validity checks after the 
programming of new edits. 


2. Ensure that the Outpatient Procedure, Revenue Code and Units of Service fields are complete 
and valid for the Inpatient (UB-92) claim types, and institute error checks to identify invalid data. 


3. Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 


4. For audit purposes, submit extract files for performance measures and encounter data in the 
requested file layouts with the requested documentation of files. 


MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described previously in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for Family Health Partners (Family Health Partners). The EQRO 
reviewed documentation between December |, 2004 and February 28, 2005. On-site review time 
was used to conduct follow-up questions, review additional documentation made available by Family 
Health Partners, and provide feedback and recommendations regarding compliance with federal 


Medicaid Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MC+ MCO, Family 


Health Partners was requested to provide the following documents on-site: 


e Member Handbook 
e Provider Administration Handbook 
e Provider Agreements 
e Grievance and Appeal Policy 
e Samples of both Grievances and Appeals related to members and providers 
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Additional documentation Family Health Partners provided on-site: 


e Sample notices to members related to plan changes, grievance, and appeal actions 

e Management reports for enrollment and member demographics 

e Winter 2005 Member Newsletter 

e Graphics for new member identification card 

e Utilization Management Plan 

e §=.- Verification of licensure for Lisa Gable 

e Grievance and Appeal quarterly reports 

e Member and Provider Grievance/Appeal Log 

e Negotiating Payment with Out-of Network Providers/Post Stabilization Services Policy 
e Methodology for Review of Education and Marketing Materials Policy (requiring and annual review) 
e Transition after Receipt of Emergency Care Policy 

e = Policy and Procedure Manual — Annual Review Requirements 


Interviews 


The following individuals were interviewed during the on-site review: 


Quality Assurance Mental Health 
Jenny Hainey, Manager, Quality Management Linda Steinke, Director of Operations 
Johanna Groves, Senior Quality Management 
Nurse Mental Health (Teleconference): 
Justin Vana, VP, Clinical Operations. CommCare 
Member Services/ Case Management Robert Young, QI Coordinator, CommCare 
/ Utilization Management 
Lisa Gable, RN, Manager, HS Plan Administration: 
Cindy Mense, Manager, Customer Service Teleconference (March 25, 2005, 1:00 p.m.): 
Patty Ornce, Senior Case Manager Bob Finuf - Chief Executive Officer 
Augusta Amadi, ER Case Manager Ma’ata Touslee, Director Health Services 


Jenny Hainey, Manager, Quality Management 
Provider Services 
Linda Steinke, Director of Operations 
Kathy Ripley-Hake, Manager, PR 


FINDINGS 

Enrollee Rights and Protections 

Family Health Partners exhibited a strong commitment to ensuring that member rights were 
protected. They had an interpreter service available if any member called and spoke a primary 
language other than English. They had a service available to translate written material into Braille if 
this was requested by a member. The Customer Service representatives, who answered member 
phone calls, set up alternatives as soon as they learned that a problem existed. The MCO utilized 
enrollment information, information gathered during welcome calls, and other collateral information 
to identify special communication needs that members had. Member Services staff worked diligently 
to ensure that members obtained the services they needed and provided any assistance possible. 
Providers were made aware of member rights and responsibilities through the Provider Manual and 


the MCO newsletter. 
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To ensure that the Member Handbook would meet the needs of MC+ Managed Care members a 
Member Advisory Committee reviewed and made suggestions prior to submission to the SMA for 
final approval. The Spanish-speaking customer service representative reviewed the Spanish version 


and made suggestions for better readability. 


Ratings for compliance with Enrollee Rights and Protections regulations (76.9%) reflected policy 
requirements that must be met to become fully compliant. A new staff person had not submitted 
the plan for annual review of Member Marketing and Educational Material. The MCO acknowledged 
that this was all together and that the policy and plan existed, but had not been sent to the SMA for 


final review and approval. 


Quality Assessment and Performance Improvement 

Access Standards 

Family Health Partners had an adequate provider network throughout the MC+ Region. The MCO 
worked to develop strong relationships with providers, particularly specialists, who agreed to serve 
MC+ members. Some would not agree to become regular network providers, but signed single case 
agreements as requested. The MCO was willing to work with out-of-network providers as 
necessary to meet members’ healthcare needs. Family Health Partners had agreements with 
Truman Medical Center and Children’s Mercy Hospital. These agreements included a 24/7 Nurse 
Advice Line to assist in after hours coverage. Members received information about this service 


through their Member Handbook, and the toll-free number was included on their identification card. 


Member and Provider Services worked as a team to resolve any member complaint. The staff 
attempted to delineate and resolve problems quickly and effectively. The MCO implemented an in- 
depth case management process to ensure that members, particularly those with special healthcare 
needs, were able to identify and access the healthcare services they needed quickly and efficiently. 
Caseloads existed for lead, asthma, catastrophic disease and obstetrics. Case Management outreach 
occurred regarding the issue of lead poisoning. This was coordinated with local health departments 


and providers. 
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A case manager was assigned to the emergency room at Truman Medical Center. The case 
manager met with members who came to Truman requesting emergency services. This case 
manager assisted members in problem solving. Referrals were often made to Family Health Partners 
case managers with special needs caseloads to assist members in obtaining appropriate services. 

The Truman Medical Center case manager ensured that members received the correct level of care, 
familiarized members with their PCP information, assisted in making appointments, and with other 


health related activities. 


Case managers involved members in care conferences with their PCPs and with hospital discharge 


planning to ensure that members were in charge of their own healthcare services. 


All provider submitted pregnancy notification forms were received by the case management 
department. Members were screened for high risk status. If the assessment indicated that case 
management services were appropriate, pregnancy plans were developed. Domestic violence was 
also a trigger for high risk pregnancy services in the MC+ population. Post-partum outreach 
occurred for new mothers. If it appeared there was any type of need, additional case management 
services were encouraged, including behavioral health services for problems such as post-partum 


depression. 


Providers were monitored regarding access to services. The MCO conducted member surveys, and 
did on-site reviews following NCQA standards. Attention was given to member records. If 
deficiencies were found a letter was sent to the provider as an explanation. A re-audit occurred. 


PCPs were visited once per month and specialists one time per quarter. 


Ratings for compliance with Access Standards (70.6%) reflected policies that required completion 
and submission to the SMA for final review and approval. The EQR recommended that completion 
of this process must occur to obtain compliance with the MC+ Medicaid Managed Care contract 


requirements and the federal regulations. 


Structure and Operation Standards 
Ratings for compliance with Structure and Operations Standards ((60%) reflected a number of 
incomplete policy requirements. Family Health Partners had a methodology for performing 


credentialing and recredentialing certifications for providers. Credentialing was reviewed every 
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three years, with annual surveys completed to monitor claims and appointment standards. All 
verification was done by the MCO following NCQA standards. Appropriate practice was in place, 
but current and correct policy remained outstanding at the time of the on-site review. The MCO 
did not have completed policy regarding excluding debarred providers from the network, policy 


regarding subcontractor relationships, and other state-required policy. 


The MCO informed members about the disenrollment process through information in the Member 


Handbook. The MCO had required disenrollment policy in place. 


Measurement and Improvement 

Family Health Partners implemented accepted clinical practice guidelines. National guidelines 
adopted by Family Health Partners began with the ACOG, for neonatal care. The MCO was part of 
the Kansas City Quality Improvement Consortium. This local healthcare quality improvement group 
reviewed and developed practice guidelines that included local standards. The MCO implemented 
local guidelines when they met or exceeded nationally accepted practices. The group currently 
adopted guidelines for diabetes and asthma, which the MCO used. The group was in the process of 
developing obesity treatment guidelines. Family Health Partners used the Milliman and Robertson 


guidelines for utilization management. 


Family Health Partners did submit two Performance Improvement Projects for validation. The full 
report can be found in the appropriate section of this report. The MCO met the criteria for 
producing complete and accurate data for Validating Performance Measures. The full report 
regarding compliance with these criteria is in the appropriate section of this report. The MCO did 
have a Health Information System (HIS) that was able to produce required information. Encounter 
Validation Data was not submitted in a format that allowed analysis. The MCO was not found to be 
fully compliant with this section of regulations. Ratings for compliance with the Measurement and 
Improvement regulations (72.7%) reflect a lack of required policy and lack of compliance with ability 


to produce required data requests. 


Grievance Systems 
Ratings for compliance with Grievance Systems regulations (100%) indicate that the MCO had 
completed all requirements regarding policy and practice in their grievance and appeal system. Files 


for both grievances and appeals, filed by members and providers, were reviewed on-site. All files 
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followed prescribed policy and timelines. Notices to members were sent within required 
timeframes and contained all required information. This information included the member’s ability 
to file a State Fair Hearing simultaneously with an MCO appeal or later if the outcome of an appeal 
was not favorable to the member. All policy and information to members included the message that 
members can maintain healthcare coverage while an appeal was pending, and explained the 


member’s responsibility to pay for charges if a decision was reached to deny the disputed service. 


The Medical Director was involved in many disputed issues and made every attempt to ensure that 
members obtained the healthcare they needed. If an appeal was filed, it was recorded in the MCO 
tracking system to ensure that another qualified individual reviewed the information submitted to 
obtain an independent appeal decision. The MCO took the grievance and appeal system very 


seriously and used information generated to inform their quality improvement process. 


Summary and Follow-up Information 

Behavioral Health Services 

Family Health Partners and CommCare developed a pharmacy education program regarding the use 
of psychotropic medications. This educational program had continued. This education program was 
provided to pharmacy personnel and to all Community Mental Health Clinics (CMHC) in the 
Region. Pharmacy representatives were present at all high volume provider group meetings where 
the providers discussed difficult issues, problem solved, and shared information. This supported the 


development of systems to identify individuals who were abusing prescription drugs. 


To expand network capacity in the area of adequate psychiatric services, the BHO identified all 
psychiatrists in the region and solicited their participation in the network. To address the shortage 
of child psychiatrists, CommCare was contracting with Advance Practice Nurses. These nurses 
were reimbursed fees comparable to psychiatrists. Further, the MCO utilized out-of-network 
providers to supplement the network. A Spanish-speaking psychologist was added. Utilization of 


behavioral health services had steadily increased. 


CommCare was attempting to impact the rate of readmission after a necessary hospitalization. 
Readmissions during 2004 decreased from previous years. Readmissions at thirty days after original 


discharge were now at approximately 8%. At ninety days after discharge the rate was approximately 
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|2%. CommCare case managers were conducting outreach by telephone within seven days of 
discharge. The aftercare plan and prescribed medications were reviewed with the member. The 
MCO reported that these interventions improved the percentage of members keeping aftercare 


appointments as well. 


All behavioral health providers sent feedback forms to PCPs with special attention to prescribed 
medication. A review of compliance with the use of the forms indicated a range of 90% compliance. 


Behavioral health case management oversight audits were done to review all intakes. 


Medical Director 


Family Health Partners had hired a new part-time Medical Director certified for pediatric and adult 


services. 


Grievance and Appeals 

Follow-up information was requested due to the number of grievances and appeals resulting from a 
lack of dental providers. The MCO believed the issue was resolved with their current dental 
subcontractor. Family Health Partners also implemented objective practice guidelines with dental 
providers. This process decreased the number of grievances and appeals. They were using the HDL 
index to score service needs and to standardize decision-making. Although some decisions were 


still overturned, the MCO reported fewer appeals and fair hearing requests. 


Family Health Partners began critically reviewing grievance and appeals during the past year. They 
wanted to ensure accuracy in reporting to the State. They were vigilant in their record keeping and 
believed the MCO achieved more accurate and consistent reporting of the issues being brought to 
their attention. The MCO closely monitored notices sent to members and reported one letter, 


which was sent outside of required timeframes. The review process occurred quarterly. 


During the past year, Family Health Partners did notice a rise in the number of grievances filed 
regarding transportation issues. This number increased due to the lack of available of car seats 
when subcontractors were transported children. Family Health Partners worked with the 
transportation contractor to provide car seats to all subcontractors. A decrease in transportation 


related grievances was the result. 
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LPHAs and School Based Clinics 

Family Health Partners was asked if they had increased their ability to work with LPHAs or school 
based clinics. They report that they did collaborate with several LPHAs, who conduct EPSDT 
examinations and lead screenings. The LPHAs met with MCO representatives and were very 
cooperative. However, they do not submit claims so the examinations they conducted were not 
captured in the Family Health Partners data system. Family Health Partners also met with the 
Kansas City School District in an attempt to engage the district in conducting school based clinics. 
The MCO offered to place a staff member at the clinic to assist in this process. The school district 


had not finalized any plans to make this an active project. 


Advisory Committee 

The MCO established a Member Advisory Committee for the review of policies and procedures, 
and to obtain input on provider services and outreach. Committee members were utilized to 
contact other members to seek input on their satisfaction with services. The MCO planned to 


continue to utilize and formalize this group to enhance service delivery. 


STRENGTHS 


|. Family Health Partners’ Provider Relations staff had developed an excellent relationship with 
physicians’ offices in the region. This enabled them to obtain timely specialist appointments for 
members for orthopedic, neurology, asthma, and other special services that are difficult to 
obtain. 


2. Family Health Partners placed a case manager on-site in the emergency department at Truman 
Medical Center. This provides members with on-site health service problem-solving. The case 
manager obtained current contact information on members and assisted them in making timely 
urgent care appointments if this was the service the member needed, rather than emergency 
medicine. The service helped members by assisting them in becoming familiar with their PCP, 
and learning how to obtain follow-up services in the most efficient manner. 


3. Family Health Partners was in the process of recruiting individuals for a Member Advisory Panel. 
Member suggestions had assisted Family Health Partners in developing a user friendly Member 
Handbook, and in making other service improvements. The MCO planned to formalize the 
process by having a standing member group as advisors to improve member input on service 
and policy issues. 


4. Family Health Partners staff was knowledgeable about MCO policy and procedures. Family 
Health Partners had an organized process for completing and submitting documentation to the 
SMA for final review and approval. 


5. Family Health Partners was launching a new method of cataloging and using their member data 
through the ManagedCare.com software. This software will assist the MCO in analyzing 
member information for project and service development. 
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AREAS FOR IMPROVEMENT 


|. Family Health Partners had not submitted an annual Member Marketing and Education Plan to 
the SMA for their review and approval. Family Health Partners staff indicated that the 
information needed is in place and would be forwarded to the SMA. 


2. Family Health Partners had met many of documentation requirements. The MCO needed 
complete this process in a timely manner. The positive practices that occurred at Family Health 
Partners should not be lost because they were not reflected in current policy. 


3. The CommCare transition policy for members engaged in a therapeutic relationship with an 
out-of-network provider did not appear to fully consider the member’s well-being. The policy 
described was four visits and then a move to an in-network therapist. Consideration of the 
member’s therapeutic needs should be a deciding factor. 


RECOMMENDATIONS 


|. Continue MCO development in the area of utilizing available data and member information to 
drive change and create opportunities for organizational growth and development. 


2. Continue focus studies and innovative community-based projects designed to meet members’ 
health and safety needs. 


3. Track required annual submissions of all documentation, such as the Member Marketing and 
Education Plan, to ensure that this occurs within required time frames. 


4. Review the CommCare provider transfer practice to ensure that members are being served 
effectively. 


5. Continue improvements in decreasing the behavioral health readmission rate. Compile data 
supporting this approach into a report that indicates continued improvement, or informs the 
BHO if there is a need to alter the plan when problems emerge. 


6. Continue reviewing MCO and provider practices through monitoring the grievance and appeal 
system. It may be useful to implement studies that produce data indicating trends that can lead 
to measurable performance improvement projects. 


7. The MCO is encouraged to continue working with school districts for the establishment of 
school based clinics. 
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FIRSTGUARD 


The previous sections of the 2004 EQRO report present the purpose and objectives technical 
methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 
Documents Reviewed 


FirstGuard supplied the following documentation for review: 


e NCQA Quality Improvement Activity Form: Identification of MC+ Children with Elevated Blood Lead 
Levels 
e NCQA Quality Improvement Activity Form: Improving Asthma Medication Management 


Interviews 

Project Leaders, participants, and the Medical Director were interviewed on-site by the EQRO 
Project Director on Wednesday, March 16, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided as were references for logic 
model development and health services research methods. The following questions were addressed 
on-site: 


Who was the Project Leader? 

How was the topic identified? 

How was the study question determined? 
What were the findings? 

What was the intervention? 

What was the time period of the study? 
Was the intervention effective? 

How do you know? 

Why or why not? 
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FINDINGS 

The first PIP evaluated was, “Improving Asthma Medication Management.” The topic or problem 
statement measured the asthma management of FirstGuard members by educating providers and 
asthmatic members and families, with the intent to improve asthma medication management. This 
represents a clinical PIP for an acute or chronic condition. Some background information was 
presented and the potential impact of improvement in asthma medication management was 
described. The study was oriented toward improving the lives of people with asthma and implies 
measures of functional status outcomes. This should be described in the narrative. The extent of 
the problem and rationale for identification was not discussed, and there was no study question 
presented. It can be assumed that better medication management has a potential for improving 
health outcome, but this should be specifically stated. It appears that all the members to which the 
study applied were included, but the rationale for age group selection was not documented. The 
study examined members from 3-56 years of age. More detailed description of the population 
within MC+ should be included, such as whether or not I 115 Waiver, 1915b Waiver, children in 
State custody, and consent decree children were all included. It should be specified that the findings 


describe outcomes of care for MC+ members. 


lt was assumed that MC+ members were targeted as part of this intervention. The study indicators 
involved identification of FirstGuard members needing asthma management and FirstGuard 
asthmatics with sub-optimal asthma medication management. Since the baseline measurement was 
conducted, FirstGuard could have included this in the presentation of rationale for the study by 
comparing the results with national benchmarks. These indicators were clearly defined and 
methods of their calculation were clearly described. The reliability and validity of the indicators is 
unknown. The method of establishing these indicators should be described. No sampling was 
conducted as part of the study. The sources of data included case management and claims data. It 
was difficult to determine whether the data were collected in a valid and reliable manner since there 
was no description of the definitions for claims (e.g., procedure codes). Measurements were 
reported to be conducted twice a year. However, the results presented indicated that 
measurements were done once a year in 2002, 2003, and 2004. The data analysis plan should be 


clearly described in the narrative. There were no barriers to the improvement strategies identified. 
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The presentation of results in tables and figures were very clear. Re-measurements were presented, 
but the intervals of re-measurement were not consistent and statistical testing was not conducted. 
Therefore, it was not possible to determine whether the intervention had a clinical or statistical 


significance. 


This study has potential for credible findings with more detail provided regarding the study topic, 
hypothesis, and study questions as well as statistical significance testing and interpretation of findings. 
FirstGuard had a good description and presentation of the data. However, major components of 
the study were not presented. Portions of the project were well-presented but perhaps because of 
the tabular format of the Quality Improvement Activity form the connection between topic, 
measurements, intervention and results was not clear. Perhaps the QIA form is not the best format 


for presenting a PIP. 


At each re-measurement, FirstGuard should document and discuss interpretations of findings, 
barriers and plan changes as a review of the potential causative factors for change or no change in 
the measures. Statistical significance testing should also be conducted, and measures should be 
added or discontinued as needed when the intervention is modified. Modify the QIA form to 


incorporate all elements of the PIP. 


The second PIP evaluated was, “Identification of MC+ Children with Elevated Blood Lead Levels.” 
The topic of the project was to achieve 100% rate of case management enrollment of MC+ children 
with elevated blood lead levels (BLL > 10 u/dL) who were successfully contacted. This is a non- 
clinical intervention that addresses the process of accessing or delivering care. Although case 
management of MC+ Members with elevated blood lead level has a significant impact on children, 
the need for improvement must be established prior to initiation of the project. This could be 
established through presenting current case management rates and nationwide benchmarks and the 
link between case management and improved health outcomes for children with high lead level 
toxicity. The goals of the study were stated, but more attention should be devoted to the 
description of the outcomes and barriers and the connection of those barriers to the intervention. 
The description of the study population should be part of the narrative. Some demographic 
information was available (0-6 year-olds), but was disjointed and presented in different areas of the 
QIA Form used to report the PIP. The selected study indicators included the percent of children 0- 


6 years of age with elevated blood lead levels who were successfully contacted, the percent of 
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children 0-6 years of age with elevated blood lead levels in case management, and the percent of 
children 0-6 years of age who were “flagged” in the case management system. The measures and 
methods of calculation were well described and sources of benchmarks were cited. There were no 


long-term outcomes clearly stated or linked to the intervention or process of care. 


There was no sampling conducted in the present study. It was not possible to determine if there 
were any particular MC+ groups excluded from the study and whether the findings represented 
results for MC+ Members only. The data collection approach only targets children who had a blood 
lead level test. Measuring the rates of blood lead testing may facilitate identification of reasons for 
change or stability in the rate of case management. It was clearly stated that FirstGuard receives 
data on children with elevated blood lead levels from the Missouri Department of Health and the 
Division of Medical Services. Other sources of data included the case management module and 
FirstGuard indicators for lead poisoning. The process for data collection and the sources of data for 
each indicator should be described. The data collection procedure appeared to be consistent, but 
validity and reliability of the measures were not addressed. The study findings were very well 
presented for the last quarter. There was no statistical significance testing conducted. Also, the 
interventions should be more clearly described so the reader is able to determine what may or may 


not have been effective about the intervention based upon the findings. 


There is some potential for valid and credible findings with the modification of measures and the 
addition of outcome measures. The topic selection was appropriate and measures were good 
indicators of the implementation of the intervention. The frequency of re-measurement periods 
was quarterly, which provides good feedback regarding the implementation of the intervention 


throughout the year. The presentation of indicators and study results was excellent. 


The explicit statement of hypotheses and study questions regarding expected outcomes and 
intermediate-term goals of attempting to intervene should suggest some measures and methods of 
measurement of the effectiveness of the intervention over and above monitoring the rates of case 
management. The QIA Form that was used for presentation of the study does have some 
limitations in that it does not allow for linking of the source of data with the actual measurement. 
This form can be redesigned or used in order to link these two concepts. The limitation of the 


form is that it is designed to describe quality improvement activity, not an actual study. It can be 
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used to help plan a PIP, however. It is recommended that this form be modified to incorporate all 


aspects of the PIP. 


STRENGTHS 


|. The measures were well described and defined for monitoring and assessment of the 
implementation of the intervention. 


2. The frequency (semi-annually and quarterly) of re-measurement periods allowed for timely 
evaluation of the effectiveness of the interventions and identification of potential issues for 
implementation. 


3. Data were presented clearly, using numerators, denominators, and rates at each measurement 
point. 


4. The topics selected are areas that are likely to benefit from the development, implementation, 
and evaluation of interventions. 


AREAS FOR IMPROVEMENT 


|. There should be additional specification of health-related outcome measures or processes of 
care that are closely associated with improved outcomes of care in the study rationale and study 
questions. 


2. Study questions, hypothesized findings, and interpretations of findings should be included in PIP 
summary and findings. 


3. Statistical significance testing between re-measurement points should be conducted to support 
any claims of effectiveness. 


RECOMMENDATIONS 


|. Include identified barriers with the implementation of the intervention, recommended changes, 
and documentation of the changes. 


2. Modify the QIA Form with sections for study question, hypotheses, and running narrative for 
results and interpretation of findings. 


3. Conduct statistical comparisons between each re-measurement and baseline to demonstrate 
improvement and stability related to the intervention. 


4. Modify the QIA Form with sections for study questions, hypotheses, and running narrative for 
interpretation of findings. 
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Validation Of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for FirstGuard. FirstGuard submitted the requested documents on 
November 23rd, 2004. The EQRO reviewed documentation between November 23rd, 2004 and 
February 28th, 2005. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by FirstGuard 
e The Baseline Assessment Tool (BAT) submitted by FirstGuard 

e MetaStar’s NCQA HEDIS 2004 Compliance Audit Report 

e Letters of communication between the EQRO and FirstGuard 

e FirstGuard’s policies pertaining to HEDIS 2004 rate calculation and reporting 
e FirstGuard’s HEDIS Project Outline for 2004 

e FirstGuard HEDIS 2004 Medical Record Review Manual 

e HEDIS 2004 Software Logs 

e CRMS data warehouse tables - data field and definitions 

e CRMS extract file build process documentation 

e MC400 Data file extracts and print screens 

e HEDIS analyzer error messages 

e | MOHSAIC data preparation process documentation 

e HEDIS data file layout in the IDS SAS Warehouse 


The following are the data files submitted for review by the EQRO: 


e Al Denom.txt 

e Al Num Data.txt 

e =6Al Numtxt 

e Asthma Denom Data.txt 

e Asthma Denom Enroll.txt 

e Asthma Num.txt 

e ADOLESCENT WELL-CARE VISITS Denom.txt 
e ADOLESCENT WELL-CARE VISITS Num Data.txt 
e ADOLESCENT WELL-CARE VISITS Num.txt 

e WC _MRR_NUMERATOR.txt 

e FileDescriptions.doc 


Interviews 

The EQRO conducted on-site interviews with Susan Richart and Arlin Crist (of Sanders software 
consulting services) at the premises of FirstGuard in Kansas City on Wednesday, March | 6th, 2005. 
They were in charge of explaining the process of calculating the HEDIS 2004 performance measures, 


as there had been a recent turnover in the HEDIS team. The objective of the visit was to verify the 
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methods and processes behind the calculation of the three HEDIS performance measures. This 
included both manual and automatic processes of information collection, storing, analyzing and 


reporting. 


FINDINGS 

FirstGuard used the Hybrid Method for Adolescent Immunization Status, Combination #I and 
Adolescent Well-Care Visits Measures. MCO to MCO comparisons of the rates of Adolescent 
Immunization Status Combination #1, Adolescent Well-Care Visits, and Use of Appropriate 
Medications for People with Asthma measures were conducted using two-tailed z-tests. For 
comparisons that were statistically significant at the 95% confidence interval (Cl), the z-score (z), the 


upper and lower confidence intervals (Cl), and the significance levels (p > .05) are reported. 


The rate for the HEDIS 2004 Adolescent Immunization Status, Combination #| reported to the 
SMA and the State Public Health Agency (SPHA) by FirstGuard was 34.55%. This was comparable to 
the statewide rate for all MC+ MCOs (14.36%; z = .34; 95% Cl: 19.69, 49.41%; n.s.). 


The rate for FirstGuard for the HEDIS 2004 Adolescent Well-Care Visits measure was 32.93%, 
which was significantly higher than the statewide rate for all MC+ MCOs (30.13%; z = .65, 95% Cr: 
29.43%, 39.19%; p = .95). The rate for FirstGuard for the 2004 HEDIS Use of Appropriate 
Medications for People with Asthma was 67.26%, which was significantly higher than the statewide 
rate for MC+ MCOs (63.92%, z = .89; 95% Cl: 63.62%, 70.90%; p > .95). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

For all three measures, FirstGuard was found to meet all criteria for producing complete and 
accurate data (see Attachment V: Data Integration and Control Findings). There were no biases or 
errors found in the manner in which FirstGuard transferred data into the repository used for 


calculating the HEDIS 2004 measures. 
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Documentation of Data and Processes 

The information systems (IS) management policies and procedures for rate calculation were 
evaluated consistent with the Validating Performance Measures Protocol. FirstGuard used NCQA- 
certified software, CareEnhance Resource Management Software (CRMS) from McKesson Inc, for 
calculation of rates for the three HEDIS 2004 performance measures. The EQRO was given a 
process overview of the MC400 claims management system, a product of OAO Health Systems, and 
was given a validation overview of the CRMS data warehouse and the Health Plan Reporter (HPR) 
module for reports. The EQRO was also given a demonstration of the data flow and integration 
mechanisms for external databases for these measures and the EQRO was also provided with a 
layout of the data structure of the internally-developed Information Delivery System (IDS) data 


warehouse for the storing of interim data. 


Data and processes used for the calculation of measures were adequate (see Attachment VII: Data 
and Processes Used to Calculate and Report Performance Measures). FirstGuard met nearly all 
criteria that applied for all three measures. The two criteria that were not met involved the use of 
Statistical significance testing to document changes in performance over time and using this 
information to support claims of improvement or stability in performance over time. Although 
many auditors conduct statistical tests to examine various aspects of data validity, such tests should 
be used by FirstGuard to assess the significance of change related to quality improvement activities 
and operational changes. The use of NCQA-certified calculation software which has been tested 
through a process of test files submitted by NCQA indicate the program specifications are adequate 


for validly calculating the rates. 


Processes Used to Produce Denominators 

FirstGuard met all criteria for the processes employed to produce the denominators of all three 
performance measures (see Attachment X: Denominator Validation Findings). This involves the 
selection of members eligible for the services being measured. 

The HEDIS 2004 Adolescent Immunization Status denominator file contained 432 denominator 
cases. This is the final sample size after a 5% oversample. There were no exclusions or 
replacements for the denominator of 411. Dates of birth and enrollment were within valid ranges, 


and there were no duplicate cases. 
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The denominator file for the HEDIS 2004 Adolescent Well-Care Visits measure contained a total of 
432 members, including a 5% over sample and the denominator of 411. There were no duplicate 


entries. The dates of service and enrollment were in the valid ranges. 


For the HEDIS 2004 Use of Appropriate Medications for People with Asthma measure, there were a 
total of 897 eligible unique members (denominator) in the ‘Asthma Denom Enroll.txt’ file. The 
‘Asthma Denom Data.txt’ file contained a list of 2,045 unique members from which the eligible 
members were derived. There were no duplicate members. The dates of birth were in the valid 


range. The dates of enrollment were not provided and could not be validated. 


Processes Used to Produce Numerators 

All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2004 criteria (see 
Attachment XIll: Numerator Validation Findings). Medical record reviews were conducted for the 


Adolescent Immunization Status, Combination #I|, and Adolescent Well-Care Visits measures. 


For the HEDIS 2004 Adolescent Immunization Status measure, FirstGuard appropriately included 
administrative events from the State Public Health Immunization Registry (MOHSAIC). The dates of 
birth and dates of service were within valid ranges. Review of the administrative hits validated |6 of 
30 administrative hits in the files provided for review by the EQRO. Fifteen (15) of 30 medical 
records requested for review were received, and |5 records resulted in valid hybrid hits. Asa 
result, the medical record review validated 56 of the | 12 hybrid hits reported. The estimated bias 
from medical record review was |3.63%. Based on the number of administrative and medical 
record review hits validated by the EQRO, the rate calculated by the EQRO was 17.52%. The total 
estimated bias for the Adolescent Immunization Status, Combination #1 measure was a | 7.03% 
overestimate of the rate. One reason the rate of bias was so high was the rate of validation of 


medical records, complicated by the submission of |5 of the 30 records. 


For the HEDIS 2004 Adolescent Well-Care Visits measure, there were |26 administrative hits, but 
only 123 reported by FirstGuard. The number of immunization events was not provided. The dates 
of service were not provided in the numerator files and could not be validated. All 18 medical 
record hits were requested and received by the EQRO for review. One additional record was 


provided for review, as it was reported that the HEDIS auditor excluded it from the numerator. 
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Twelve (12) of the records were validated, resulting in a bias of 1.46% from the medical record 
review. Of the 19 numerators, all 19 showed evidence of a physical examination, | 7 showed 
evidence of a medical history, and |2 showed evidence of anticipatory guidance. A total of 138 of 
the I4I hits were validated, for a rate of 33.58%, a final overestimate of ./3%. 

The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was the third 
measure validated. There were a total of 606 unique members found in the file for 602 reported by 
FirstGuard. Two members in the numerator file did not exist in the given denominator file and 
could not be validated. All the numerator events could not be identified using CPT or UB-92 
codes. The dates of birth and service were within the valid ranges. The final rate was calculated to 


be 67.49%, underestimate of .22%. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Adolescent Immunization Status and Adolescent Well-Care 
Visits. Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: Sampling 
Validation Findings were completed for each of these measures. There were no exclusions 
calculated for the Adolescent Immunization Status measure. None were allowable for the 


Adolescent Well-Care measure. 


Submission of Measures to the State 

FirstGuard submitted the DST for each of the three measures validated to the SPHA, (the Missouri 
Department of Health and Senior Services; DHSS) in accordance with the Code of State Regulations 
(19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 


Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

Table | summarizes the estimated bias in the rates reported by FirstGuard, with the direction of 
bias. Although data processes and procedures for calculating the Adolescent Immunization Status, 
Combination #| measure were adequate, the rate calculated by the EQRO as a result of validating 
the medical record review fell outside the 95% confidence interval reported by FirstGuard. The 
large overestimate is likely due to the small proportion of medical records submitted for review. It 


is not possible to assume that the medical records not submitted contain evidence of immunizations. 
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Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures. 


Measure Estimate of Bias Direction of Estimate 
Adolescent |!mmunization Status, Combination #1 17.03% Overestimate 


Adolescent Well-Care Visits 13% Overestimate 
Use of Appropriate Medication for People with 
Asthma 0.22% Underestimate 





Final Audit Rating 
The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 


measure. 


Table 2. Final Audit Rating for HEDIS 2004 Performance Measures. 


Measure la latclMcNereliam»t-halate 
Adolescent Immunization Status, Combination #1 Not Valid 


Adolescent Well-Care Visits Fully Compliant 

Use of Appropriate Medication for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by the MCO. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. FirstGuard was fully compliant with state specifications for the measurement of the HEDIS 2004 
Adolescent Well-Care Visits and Use of Appropriate Medications for People with Asthma 
measures. 


2. FirstGuard demonstrated significantly higher rates of Adolescent Well-Care Visits and Use of 
Appropriate Medications for People with Asthma than the average for all MC+ MCOs. 


3. The rates for HEDIS 2004 Adolescent Well-Care Visits was higher than the National 
Commercial average for the measure. 


4. There was excellent documentation of policies and procedures for calculation of rates for 
HEDIS measures. The documentation enabled proper replication of processes during a period 
of employee turnover. FirstGuard had a good layout for the HEDIS project plans and integrated 
it with documents within the network with incorporated hyperlinks. 


5. FirstGuard used NCQA-certified software, CRMS/HPR of McKesson, for calculation of HEDIS 
measures. This software was reviewed and tested by NCQA auditors for source code 
verification. There were effective CRMS validation processes in place. 


6. There was effective use of the State Public Health Immunization Registry (MOHSAIC) data for 
calculation of the immunization measures. The data are directly loaded with mapping of data 
fields into the CRMS warehouse, which then processes it for reporting in the HPR module. 


Performance Management Solutions Group 323 
8. HC a Division of Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


7. There was an efficient data integration process in place. Third party administrator StratumMed 
and clearinghouse WebMD’s data were integrated into the claims system. There were some 
home-grown codes (not relevant to the audited measures) effectively mapped within the claims 
system. The MC400 claims management data that is relevant for the HEDIS performance 
measures was loaded into the IDS data repository without loss of data. 


8. There were effective data validation processes with numerous checks for errors and capacity 
for system edits within the claims system and the reporting software CRMS. The IDS system 
had an error process for identifying discrepancies in an error file which was reviewed both 
manually and within the system. FirstGuard also conducts periodic quality reviews. Reject 
rates, export counts, import counts and data integrity issues are reviewed by FirstGuard along 
with McKesson for validity purposes. 


9. In response to preliminary findings, FirstGuard provided supplemental information regarding the 
process of identifying hybrid or administrative numerators and the rationale (cost and security 
of member privacy) for not retaining medical records on-site. 


AREAS FOR IMPROVEMENT 


|. Data analysis should incorporate tests of statistical significance to assess whether the observed 
changes in rates are related to a specific intervention. There is an opportunity to include tests 
of significance to identify changes or stability from year to year. 


RECOMMENDATIONS 
|. FirstGuard should conduct and document statistical comparisons on rates from year to year. 


2. Continue with the documentation processes in place. It makes it easier for regulatory 
compliance and effective information management within the organization. It also serves as a 
knowledge tool for data management. 


3. Continue with periodic quality checks and reviews of the data integration process. It eliminates 
data errors and discrepancies. 


4. Continue using the NCQA-certified software for calculation of HEDIS measures as it is a 
compliant tool for reporting. 


5. Submit extract files of numerators and denominators for performance measures with clearly 
identified indicators for administrative and hybrid hit to allow for medical record sample 
selection and measure validation. 
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Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data validation in 
the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical fields? 


For the Medical claim type, there were a total of 98,367 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Outpatient Recipient ID field was |00.00% complete, accurate and valid. 


3. The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.97% valid. 
Invalid dates of service ranged from 09/18/2003 — 12/31/2003. 


4. The Outpatient Last Date of Service field was 100.00% complete and accurate, and 99.98% valid. 
Invalid dates of service ranged from 04/02/2004 — 06/24/2004. 


5. The Outpatient Units of Service field was 100.00% complete, accurate and valid. 


6. The Outpatient Procedure Code field was |00.00% complete and accurate, with 96.99% 
containing valid values. The 2,959 invalid procedure codes consisted of | 7 “B4035”, 2 “B4036”, 
and 2,940 “Y0043” entries. 


7. The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
8. The first Diagnosis Code field was |00.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 42.35% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 18.32% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
8.13% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. 


For the Dental claim type, there were a total of 8,866 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, 


accurate and valid. 
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For the Home Health claim type, there were a total of 14 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004. All fields except the second through fifth Diagnosis 


Code fields examined were |100.00% complete, accurate and valid. The second, third, fourth, and 


fifth Diagnosis Code fields fell well below the 100% threshold for completeness, accuracy, and 


validity established by the SMA (0.00% for each). 


For the Inpatient claim type, there were a total of 12,688 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


a ow 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 100.00% complete, accurate and valid. 


The Admission Date field was |00.00% complete and accurate, and 96.05% valid. There were 
501 invalid dates ranging from 12/05/2003 — 12/31/2003. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 99.16% (with 106 entries of 
99999999). Valid values were present 94.99% of the time. In addition to the invalid 
“99999999” entries, 656 invalid dates ranged from 04/01/2004 — 05/17/2004. 


The Bill Type field was 100.00% complete, accurate and valid. 


The Patient Status field was 100.00% complete with the correct number of characters (size). 
This field requires two digits. The Patient Status field was 99.93% accurate and valid. There 
were nine invalid codes of “63”. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (94.11%, 79.40%, 65.64%, 51.76%, 
respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


. The First Date of Service field was |00.00% complete and accurate, and 96.05% valid. There 


were 50 invalid dates of service ranging from 12/05/2003 — 12/31/2003. 


. The Last Date of Service field was 100.00% complete and accurate, and 94.83% valid. There 


were 656 invalid dates of service ranging from 04/01/2004 — 05/17/2004. 


. The Revenue Code field was 99.99% complete, accurate, and valid. One field was blank 


(incomplete, inaccurate, invalid). 


. The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were a total of 52,998 encounter claims paid by the 
SMA for the period January I, 2004 through March 31, 2004. 


The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
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The Recipient ID field was 100.00% complete, accurate and valid. 
The First Date of Service field was 100.00% complete and accurate, and valid. 
The Last Date of Service field was 100.00% complete and accurate, and valid. 


The Units of Service field was 100.00% complete, accurate and valid. 


Se we 


The Outpatient Procedure Code field was 100.00% complete. There were 25,601 invalid 
procedure codes ranging from “0220” — “0983”, resulting in an accuracy and validity rate of 
51.69% for this field. The Outpatient Procedure Code field requires a five-digit alphanumeric 
sequence. 


7. The Outpatient Revenue Code field was 100.00% complete, accurate and valid. 
8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 54.64% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 24.59% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
10.96% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 5.52% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were a total of 72,990 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. All fields examined were |00.00% complete, accurate and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for FirstGuard, an error analysis of the invalid entries was conducted 
for fields which were lower than the |100.00% threshold specified by the SMA. For the medical claim 
type, the Outpatient Procedure Code field contained invalid codes (see previous findings). For the 
Inpatient Claim type, there were invalid codes in the Discharge Date, Patient Status fields, and one 
blank in the Revenue Code field. There were invalid codes in the Outpatient Procedure Code field 


for the Outpatient Hospital claim type. 
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What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the rates of Medical, Home 
Health, Inpatient, Pharmacy, and Outpatient Hospital claim types were consistent with the average 
for all MC+ MCOs. The rate of Dental claim types were significantly higher for FirstGuard than the 
average for all MC+ MCOs. These findings suggest a high level of completeness of data and at least 


moderate access to and utilization of services. 


The findings from the performance measure analysis indicate that FirstGuard identified a consistent 
rate of administrative hits for the HEDIS 2004 Adolescent Immunization Status, Combination #1 and 
Adolescent Well-Care Visits measures compared to the average for all MC+ MCOs, while there 
was a significantly higher rate of administrative hits for the Use of Appropriate Medications for 


People with Asthma measure. 


To What Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate Between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 
the period of January |, 2004 through March 31, 2004 for medical record review. Of the 95,566 
Medical encounter claim types in the SMA extract file for January |, 2004 through March 31, 2004, a 
total of 100 encounters were randomly selected. This was an oversample, as the minimum required 
sample size was 88 per MC+ MCO. Providers were requested to submit both medical records and 
claim forms or claim histories for review. There were 7/4 medical records (74.0%) and I5 claim 
forms (15.0%) submitted for review. Encounters for which no documentation was submitted were 
unable to be validated. The match rate for procedures was 49.0%, with a fault rate of 51.0%. The 


match rate for diagnoses was 56.0%, with a fault rate of 44.0% 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing information (n = 86) 
or incorrect information (n = 2). For the diagnosis description in the medical record, the reasons 
for diagnoses not matching the SMA extract file were missing or illegible information (n = 49), or no 


Performance Management Solutions Group 328 


a Division of Behavioral Health Concepts, Inc. 


2/16 Forum Blvd., Suite 4 T: 573.446.0405 
Columbia, MO 65203 F: 573.446.1816 
www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


match with the description of the symptoms based on the information in the medical record (n = 5). 
For the diagnosis code on the claim form, the reason for diagnosis codes not matching the SMA 
extract file was missing or illegible information (n = 87). The reason for the diagnosis descriptor on 


the claim form not matching the SMA extract file included missing or illegible information (n = 71). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = |2), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; (n = |), incorrect codes (n = 2), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = 8), and not enough information (n = 3). For the procedure code on the claim form, the 
reason for procedure codes not matching the SMA extract file was missing or illegible information 
(n = 90). For the procedure description on the claim form, the reason for procedure codes not 


matching the SMA extract file was missing or illegible information (n = 96). 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, FirstGuard had difficulty submitting files in the requested file layouts for 
the encounter data validation processes. Files were requested in flat file (comma-delimited), 
machine readable format. Although flat file format was submitted, there were several files (unpaid 
claims) that were not able to be loaded for analysis. The following section summarizes the issues 


found when attempting to load files submitted for encounter data validation analysis. 
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What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


No documentation was provided for the files submitted. Subsequent communication from 
FirstGuard indicated that unpaid claims could not be submitted in the national standard file layout 


requested. 


I. BHC Paid Enctrs.txt |,.130 KB Text Document [2/29/2004 3:17PM 
This file contained 3 file layouts: Records | - 4780 were in NSF/CMS 1500 file layout; records 4781-6422 were in 
UB-92 file layout; and records 6423 - 6990 were in NCPDP file layout. For the NCPDP file layout, 568 of 6,423 


records were able to be loaded for analysis. 


2. BHC Paid Enctrs HCFA.csv Il1 KB = Microsoft Excel Comma Separated Values File 12/30/2004 3:15 PM 


This file was not in any known file layout and was unable to be loaded for analysis. 


3. BHC Unpaid Enctrs UB92.csv 8,815 Microsoft Excel Comma Separated Values File 12/30/2004 3:15 PM 


This file was not in any known file layout and was unable to be loaded for analysis. 


STRENGTHS 


|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 


2. The critical fields evaluated for the Dental, Home Health, and Pharmacy claim types were 
|00.00% complete, accurate, and valid. 


3. The match between the SMA encounter claims data and the medical records for diagnosis and 
procedure codes was consistent with the average for all MC+ MCOs. The primary reasons for 
errors were missing and illegible information. 


4. The rate of each claim type was consistent with or higher than the average for all MC+ MCOs, 
and the rate of administrative claims found for the performance measures suggest at least 
moderate levels of encounter data completeness. 


AREAS FOR IMPROVEMENT 
|. For the Medical claim types, the Outpatient Procedure code field contained invalid codes. 
2. For the Inpatient claim type, the Discharge Date and Patient Status fields contained invalid data. 


3. For the Outpatient Hospital claim type, the Outpatient Procedure Code field contained invalid 


values. 
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RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the NSF/CMS 
1500 file layout for the Outpatient Procedure Code and run validity checks after the 
programming of new edits. 


2. Ensure that Discharge Date and Patient Status fields are complete and valid for the Inpatient 
(UB-92) claim types, and institute error checks to identify invalid data. 


3. Institute edits and error checks for the Outpatient Hospital claim type Procedure Code field 
(UB-92). 


4. For audit purposes, submit extract files for performance measures and encounter data in the 
requested file layouts with the requested documentation of files. 


MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described previously in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for FirstGuard. The EQRO reviewed documentation between 
December |, 2004 and February 28, 2005. On-site review time was used to conduct follow-up 
questions, review additional documentation made available by FirstGuard, and provide feedback and 


recommendations regarding compliance with federal Medicaid Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MCO, First Guard 
was requested to provide the following documents on-site: 


e Member Handbook 

e Provider Handbook 

e Provider Agreements (last review date 1998) 

e Member Marketing and Education Plan 

e Sample of Grievance and Appeal files representing both members and providers 


Additional documentation provided by FirstGuard on-site: 


e Member Grievance Category Trends report 

e Member Appeals Category Trends report 

e Provider Review Process policy 

e MC+ Marketing Materials policy 

e Member Grievance flow chart and policy 

e Grievance System IT information 

e = FirstGuard 2004 Quality Improvement Annual Program Evaluation 
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Interviews 


The following individuals were interviewed during the on-site review: 


Plan Administration 

Jean Rumbaugh, Chief Operating Officer 

Dr. William Pankey, VP, Clinical Quality 

Barbara Maloney, VP, Development and Regulatory 
Affairs 

Celia Humphreys, Director, QI and Credentialing 
Sandra Claussen, Project Manager 


Quality Assurance 

Celia Humphreys, Director, QI and Credentialing 
Sandra Claussen, Project Manager 

Dr. William Pankey, VP, Clinical Quality 

Barbara Maloney, VP, Development and Regulatory 
Affairs 


Member Services/ Case 

Management/ Utilization Management 
Jean Rumbaugh, Chief Operating Officer 
Jackie Jones, Director, Medical Management 
Dr. William Pankey, VP, Clinical Quality 
Sharon Traylor, Director, Customer Care 


FINDINGS 


Enrollee Rights and Protections 


Provider Services 

Dr. William Pankey, VP, Clinical Quality 

Barbary Maloney, VP, Development and Regulatory 
Affairs 

Sandra Claussen, Project Manager 

Celia Humphreys, Director, QI and Credentialing 


Mental Health 
Dr. William Pankey, VP, Clinical Quality 
Sandra Claussen, Project Manager 


Teleconference with Magellan Behavioral 
Health: 

Julie Billingsly, Account Manager 

Thomas Hamlin, Corporate Medical Director 
Lawrence Miller, Compliance Officer 

Jonathan Miller, QI 


Ratings for compliance with the Enrollee Rights and Protections regulations (100%) reflected 


FirstGuard’s commitment to its members and to completing policy and procedure as required. The 


MCO exhibited a firm commitment to ensuring that members had access to and obtained all of the 


healthcare services they needed. MCO staff stressed during interviews that showing respect and 


dignity to members begins with the first contact with the organization. During all telephone 


contacts, which included Welcome Calls or the first inquiry, members were treated with respect. 


FirstGuard staff shared the importance of always discussing members with the same respect that 


was shown when speaking to them directly. The MCO explained that their policy and practice 


required that everyone from the members themselves, to providers, and their own staff members 


were trained on what constitutes discrimination, cultural competency, and minority inclusion. 


Training included the mechanism for reporting any situation where a member believed they were 


“discriminated against or have witnessed an act of discrimination.” 


The MCO provided translation services so that communication was not a barrier to obtaining 


healthcare for members. FirstGuard had staff that spoke Spanish and was actively recruiting for a 
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staff member who spoke Vietnamese. They had the capacity to have all written material translated 
into Braille, or to produce a tape or CD version of the Member Handbook or any other member 
information. The MCO had access to the Missouri Relay Service to communicate with members 


who were hearing impaired. 


Quality Assessment and Performance Improvement 

Access Standards 

FirstGuard had a complete network throughout this MC+ Managed Care Region. The observation 
was shared that the MCO had specialists and PCPs within the network in some of the more rural 
areas of the Region. The MCO reported that Provider Relations worked diligently to develop an 
honest and strength-based relationship with providers. This created a loyalty and willingness to 
provide services to MCO members. Provider Relations staff strictly monitored providers to ensure 
that care was provided in the prescribed timeframes. They also asked that access surveys were 
completed to ensure that members had the availability of required services. FirstGuard willingly 
utilized out-of-network providers if this was the best method to meet the needs of a specific 
member. Examples included using the Children’s Hospitals in Texas and Arkansas for services 
within the past year. The MCO also sent one member to San Francisco for repair of a congenital 
heart defect. This required several follow-up trips until the member was stable enough to be seen 


by the local provider. 


FirstGuard had an active case management program. They attempted to provide some care 
coordination for any member who needed it, not only those with identified special healthcare needs. 
The case managers monitored provider communication and did follow-up on all members who 
received mental health services to ensure coordinated communication between providers. 
FirstGuard case managers made a concerted effort to identify, and make referrals if necessary, to all 
services that benefit members including those offered through public health agencies, the school, 
school-based clinics or other community-based caregivers. Case management staff made every 
effort to identify members who needed a specialist as a PCP. The MCO had a mechanism to locate 
and place members in a specialist's care if that provider was the most appropriate to become the 


member’s PCP. 
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Ratings for compliance Access Standards (88.2%) reflected one documentation issue to be resolved. 
The SMA required the MCO to obtain full compliance with the access standard regulations. The 
MCO was required to update their subcontractor agreements. This was done through an 
amendment process, which was completed at the time of the on-site review. This amendment had 


not been submitted to the SMA for their review and approval. 


Structure and Operation Standards 

Ratings regarding compliance with the Structure and Operation Standards (90%) reflect that 
FirstGuard is substantially compliant with these regulations. The MCO had approved stringent 
credentialing and disenrollment policies. All practice within the operations outlined in this section 
met regulatory standards. The unresolved issue was the completion and submission of the 
amendment to subcontractor agreements. When this issue is resolved with the SMA, FirstGuard 


will be fully compliant with these regulations. 


Measurement and Improvement 

FirstGuard actively participated in the Kansas City Quality Improvement Consortium, which 
developed clinical guidelines for asthma, diabetes, and other medical conditions. The MCO used 
nationally accepted guidelines where appropriate. The MCO disseminated and monitored 
application of these guidelines through provider training and posting information on the MCO 
website. The MCO believed that their one-on-one efforts with providers assisted in the acceptance 


of the national and local guidelines. 


FirstGuard had an active Quality Assessment and Improvement program. The MCO shared its most 
current program evaluation during the on-site review. This program was composed of an internal 
system of monitoring, analysis, evaluation and improvement in the delivery of healthcare that 
included care by all providers. Staff with expertise in quality assessment, utilization management, 
and continuous quality improvement performed monitoring activities. This information was used to 
drive system change. The results of the monitoring activities were shared with the MCO board 
quarterly. This helped to ensure that quality improvement activities were put in place as the result 


of the monitoring efforts. 
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The MCO did provide two Performance Improvement Projects for validation. The results are 
located in the appropriate section of this report. FirstGuard used the hybrid method for validation 
of two performance measures. The results of the medical record review for the performance 
measure validation is contained in the appropriate section of this report. FirstGuard does have a 
health information system (HIS) capable of meeting the MC+ Managed Care program requirements. 
The information supplied for the Validating Encounter Data was not submitted in a format that 


allows for analysis of this data. 


Ratings for compliance with the Measurement and Improvement regulations (63%) reflect the need 


to improve data submission. 


Grievance Systems 

Ratings for compliance with Grievance Systems regulations (94.4%) reflect that the MCO was 
substantially compliant regarding policy and practice of their grievance and appeal system. Files for 
both grievances and appeals, filed by members and providers, were reviewed on-site. All files 
followed prescribed policy and timelines. Notices to members were sent timely and contained all 
required information. This information included the member’s ability to file a State Fair Hearing 
simultaneously with an MCO appeal, or later if the outcome of an appeal was not favorable to the 
member. All policy and information to members included the message that members can maintain 
healthcare coverage while an appeal was pending, and explained the member’s responsibility to pay 
for charges if a decision was reached to deny the disputed service. The Medical Director was 
involved in many disputed issues and made every attempt to ensure that members obtained 
healthcare when needed. If an appeal was filed the MCO maintained a tracking system to ensure 
that another qualified individual, not involved in the original decision, reviewed the information 
submitted to obtain an independent appeal decision. The MCO took the grievance and appeal 
system very seriously and used information generated to inform their quality improvement program. 
FirstGuard developed a database to effectively track grievance and appeals. This system was 
designed to track grievances and appeals to ensure that the MCO met all required timeframes. 
The MCO has also initiated an Appeal and Grievance Committee. This group included members 
from all internal departments. This committee analyzed available data from grievances and appeals 


and identified areas for program improvement. 
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The issue that remains outstanding is the completion and approval of the amendment to 
subcontractor agreements. When this issue is resolved with the SMA, FirstGuard will be fully 


compliant with these regulations 


Summary and Follow-up Information 

Behavioral Health 

There were a number of follow-up issues regarding mental health. Most of these were no longer 
pertinent as FirstGuard will be changing to a new behavioral health system during the coming year, 


as the result of new MCO ownership. 


The issue that remains outstanding is the completion and approval of the amendment to 
subcontractor agreements. VVhen this issue is resolved with the SMA, FirstGuard will achieve full 


compliance with these regulations 


Magellan, the contracted BHO for the review period, was interviewed. They were asked about the 
Post-Partum Depression Study that was started at the time of the previous audit. Magellan staff 
explained that this was performed by their Chicago Office. When management moved to Dallas, 


the study was abandoned because the information system could not support it. 


School-based Services 
FirstGuard reported that they attempted to engage the Kansas City School District in trying to 
methods, such as a school-based clinic to increase the numbers of preventive health screenings, 


including EPSDT. The district was not responsive. 


Impact of the CLAS (Culturally and Linguistically Appropriate Services) Workgroup 

The CLAS workgroup had determined that FirstGuard did well in providing services to the Spanish 
speaking community. The MCO materials were translated and they had Spanish-speaking staff 
available. The MCO contracted with several medical clinics that included Spanish-speaking 
physicians. In analyzing member survey information, enrollment and other demographic information 
the CLAS workgroup found there were no significant problems or gaps in terms of sensitivity to and 


knowledge about culturally impacted healthcare issues. 
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lt was noted that in a case discussed during the on-site review, FirstGuard staff described a situation 
where a member was in serious need of an obstetric provider who spoke Mandarin Chinese. A 
provider was located. This physician provided obstetrical services, and agreed to provide follow-up 


pediatric services to the member and her family. 


FirstGuard provided staff training to ensure that smaller groups who might encounter problems, 


were treated with respect, understanding, and received adequate healthcare services. 


Provider Relations 
In the past FirstGuard has delivered informational presentations to providers and their staff. These 
presentations were found to be successful in educating and informing providers about changes and 


new clinical issues. The MCO planned to reintroduced these sessions during 2005. 


STRENGTHS 


|. FirstGuard had developed an extensive provider network. This included provider agreements in 
the small rural communities within their MC+ Region. FirstGuard’s network included physicians 
in specialty areas that have been difficult to recruit. They have done this through consistent 
service and strong relationship building. 


2. FirstGuard developed a project to improve birth weight outcomes for members who have high- 
risk pregnancies. They used a multi-disciplinary approach, and were looking at the psychosocial 
issues that research indicated negatively impacted birth weight. 


3. FirstGuard staff demonstrated a commitment to quality integration throughout their 
organization during the entire site visit. 


4. FirstGuard utilized an interdisciplinary committee to review grievances and appeals. This group 
reviewed trends and data to ensure that service improvements were implemented timely and 
effectively. 


5. FirstGuard maintained oversight of services provided by Magellan Behavioral Health. Magellan 
was located out-of-state. FirstGuard case managers were involved to ensure that members 
received appropriate services and that PCP’s were informed of the mental health services their 
patients received. 
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AREAS FOR IMPROVEMENT 


FirstGuard will be transitioning members to a new behavioral health organization during the 
coming year. It is imperative that the MCO maintains oversight of this process to ensure that 
members do not lose services, and continue receiving appropriate levels of service. 


FirstGuard had just completed obtaining addendums to their subcontracts. Ensure that required 
documentation is submitted to and approved by the SMA for final review and approval. 


RECOMMENDATIONS 


Continue completion of documentation with the same level of importance observed in the high 
standards reflected in the daily practice within the MCO. 


Continue MCO development in the area of utilizing available data and member information to 
drive change and create opportunities for organizational growth and development. 


Work closely with the new BHO to ensure a healthy transition process for members. Develop 
methods to enhance mental health service delivery. Work with the new BHO to develop studies 
and to communicate concerns that can lead to improved performance. 


FirstGuard stated a willingness to re-approach the Kansas City School District to collaborate on 
opportunities to improve adolescent health screening rates. FirstGuard is encouraged to 
continue this endeavor. To meet the needs of the MC+ population the MCO may wish to 
consider engaging another district that includes a significant number of FirstGuard members. By 
demonstrating the effectiveness of this approach, the MCO may then be able to convince the 
Kansas City District of the benefits to their students.. 


Continue explicit efforts to be aware of new cultural issues within the community. Continue 
responsive and proactive focus in informing staff and providers regarding cultural sensitivity. 


Continue to utilize available data to analyze grievance and appeals and to identify trends to 
create opportunities for program and process improvement. 


Utilize the provider presentations to educate providers about clinical guidelines and the benefits 
of physician profiling. 
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BLUE ADVANTAGE PLUS 


The previous sections of the 2004 EQRO report present the purpose and objectives technical 


methods, procedures for evaluation, MCO to MCO comparisons for all MC+ MCOs on analyses, 
and findings and recommendations for each of the protocols based on the aggregate analysis. This 
section of the EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed Care 
Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


Performance Improvement Projects 


METHODS 
Document Review 


Blue Advantage Plus supplied the following documentation for review: 


e NCQA Quality Improvement Activity Form: Improving Care for Asthmatics 
e NCQA Quality Improvement Activity Form: Lead Testing Improvement Project 


Interviews 

Project Leaders, participants, and the Medical Director were interviewed on-site by the EQRO 
Project Director on Tuesday, March |5, 2005 to review the methods, study design, and findings. 
Technical assistance regarding study design and measures was provided, as were references for logic 
model development and health services research methods. Specific questions regarding each study 
were addressed: 


Asthma 

What is a baseline goal? 

Measures #1 and #2 indicate, “...in the total asthma population?” How is this defined? 

Regarding the “sample”, how many were sampled, how many participated? How many responses were able 

to be analyzed? What was the size of the population? 

What statistical tests were used, and what were the degrees of freedom? 

To what were the significant findings from baseline and re-measurement attributed? 

To what were the non-significant findings from re-measurement to re-measurement attributed? 

What was the rationale for conducting statistical significance testing from baseline to re-measurement on 

some measures and re-measurement to re-measurement on other measures? 

e What interventions were implemented and when? 

e How many letters were sent to members and how many were not returned due to undeliverable 
addresses? 

e What findings are relevant to the MC+ population? 
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e Describe how flu shots and depression screening relate to asthma and the measures. 
e How was symptom improvement analyzed? 
e What are the conclusions about the effectiveness of the intervention(s)? 


e Lead Testing 

e What was the rationale for using six months continuous enrollment? 
e What is the “specified time period?” 

e Why were measures restricted to Jackson County? 

e Was there an intervention targeted to Jackson County? 


FINDINGS 

The first PIP evaluated was, “Lead Testing Improvement Project.” The topic was to improve lead 
testing rates for |2- to 72-month-old Blue Advantage Plus members, which was focused on 
preventing an acute or chronic condition. The need for improvement was presented and supported 
in the narrative. Although Blue Advantage Plus had not conducted data collection and analysis to 
support the rationale, secondary data supported rationale for selection of the topic. The project 
addressed a broad range of key aspects of enrollee care and services by evaluating the effectiveness 
of provider and member education in increasing blood lead testing rates. It was assumed that MC+ 
Managed Care Members were included in the study, however, more detailed description of study 
participants should be provided. Additionally, the rationale for selecting one county was not well- 
supported and does not map to the 0 to 6-year-old age range or | to 6-year-old age range indicated 
in the narrative. The rationale for this should be specified. There were no actual questions 
addressed in the Quality Improvement Activity (QIA) Form. Appropriate questions would have 
been, “Does education of providers and members about blood lead testing improve lead testing 
rates among |2- to 72- month-old Blue Advantage Plus members; and “Does this improve the rates 
of lead testing among children 6 years of age and younger in a high-risk area?” The indicators were 
very well-defined, as well as their benchmarks and sources and baseline goals. They included the 
percent of |2-month-olds in Blue Advantage Plus service area who had a lead test and at least six 
months continuous enrollment; percent of 24-month-olds in Blue Advantage Plus service area who 
had a lead test and at least six months continuous enrollment; percent of |2-month-olds, 24-month- 
olds, 36-month-olds, 48-month-olds, 60-month-olds, and 72-month-olds residing in Jackson County 
who had a lead test and at least six months continuous enrollment. The association between the 
intervention measures and outcomes is implied and apparent, but it would be beneficial to describe 


it more clearly in the study. 
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The age range of members was clearly defined in the study topic, question and indicators. However, 
a more detailed study of participants, such as an indication that findings relate to MC+ Members 
specifically, and that all MC+ members (e.g., || 15 Waiver, 1915b Waiver, children in State custody, 
and consent decree children) were included in the study. There was one county selected on the 
basis that it had the greatest concentration of children in the Blue Advantage Plus service area. No 
other sampling was conducted. The sources of data were clearly defined and the study design 
specified systematic method of collecting valid and reliable data over time. There was no data 
analysis plan identified in the narrative. This could be included in the section on how the analysis 
will be conducted, for example, “‘z scores will be calculated on measures | through 5, comparing the 


baseline rate to the re-measurement rate to assess the impact of the intervention”. 


The intervention included the education of members, reminder letters for EPSDT testing, supplies of 


filter paper to providers. There was a good description of analysis of barriers. 


Although there was information regarding re-measurement points, graphs could more clearly 
indicate when re-measurement was done so they are self-contained sources of information. The 
analysis did identify initial and repeat measurements, statistical significance, and factors that influence 
initial and repeat measurement. There should be more discussion of the analysis and presentation 
of confidence intervals. The level of significance should also be presented. A 95% confidence level 
would be sufficient for this type of study. The same methodology as baseline measurement was 
used when measurement was repeated. Although there was no statistically significant 
improvement, there was no decline, suggesting stability in the measure. The data presented were 
preliminary re-measurements at the first re-measurement point. Therefore, it is difficult to assess 


sustained improvement until additional re-measurement points are instituted. 


This was a very well-designed and implemented study with high potential for producing credible 
findings. There was a good study topic, a well-developed rationale and nice presentation of results. 
Although there was an adequate description of the interventions and barriers, there was no 
description of the efficacy of intervention with regard to the measurements that were done. The 
possible reasons for no significant improvement at the first re-measurement point should be 
discussed and barriers and changes to the study or intervention should be suggested. 

In the future, specify clearly the study questions and devote more attention to the description of 


sampling methodology. Outline the data analysis plan in detail and identify statistical tests conducted 
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or planned. After re-measurement, evaluate the effectiveness of the study and interventions. QIA 
Form used to document the study findings may need to be adapted to incorporate some 
recommendations. Include process measurements of the intervention (e.g., the number of provider 


medical records in which a verbal lead screen or a BLL are documented). 


The next PIP evaluated was, “Improving Care for Asthmatics.” The topic and problem statement 
was to improve asthma management, which represented addressing a clinical issue involving care for 
an acute or chronic condition. There was a well-developed presentation of study rationale and 
topic statement. To support the need for improvement, it is important to compare, for example, 
prevalence rates of asthma in Blue Advantage Plus Members to national benchmark or comparative 
state data. Multifaceted interventions were developed and implemented to improve care and 
functional outcomes for people with asthma, addressing a broad spectrum of key aspects of enrollee 
care and services. The demographic characteristics of the study population should be clearly 
presented in the narrative. Some information can be inferred, but is scattered throughout the QIA 
Form. The age groups defined were from 2 to 56 years of age. On-site review indicated study 
findings were specific to MC+ Members and that no specific groups were excluded from the 
intervention. All MC+ Members had equal opportunity to participate. The study question appeared 
to be, “Does the implementation of an asthma disease management program improve the functional 
outcomes of participants and the use of pharmacy and acute care services among the asthmatic 


MC+ member population as a whole?” 


The study used objective, clearly-defined indicators and targeted goals for the number of inpatient 
admissions for the care of asthma per 1,000 members, the number of ER visits for the care of 
asthma per |,000 members, percent of Blue Advantage Plus members identified with persistent 
asthma that are prescribed medications acceptable as primary therapy for long-term control of 
asthma, functional status (frequency of nightmare awakenings). The titles of these indicators were 


very well detailed and suggested methods of measurement. 


A more detailed description of study participants should be presented to specify whether the study 
findings related to MC+ Members, and whether all MC+ member groups (1115 Waiver, 1915b 
Waiver, children in State custody, and children in the consent decree). It appears no sampling was 
conducted and that “all eligible members” were included in the personal interview for one of the 


measures. There does seem to be some misconception about sampling. One statement indicated 
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“the sample size was determined by how many people we were able to contact during the quarter 
being measured”, which describes a participation rate or level of completeness of the data collected 
rather than a sample. However, the response rates are useful for measuring of the effectiveness of 
the implementation of the intervention. No actual sampling was conducted. There were excellent 
descriptions of data, data sources, and methods and procedures for data collection from multiple 
sources (members and claims). The study also designed a systematic method of collecting valid and 
reliable data, but no survey tools were provided for review. There should be a standard member 
survey and this should be included in any descriptions of the PIP. There was not an analysis plan 
identified in the narrative. Simply stating that “z-scores will be calculated on measures | through 5, 


comparing the baseline rate to the re-measurement rate” would be sufficient. 


Blue Advantage Plus presented an excellent analysis of barriers encountered in the process of 
implementing the intervention. Actions undertaken seem to be adequate to address the barriers. It 
is clear from the narrative which intervention was designed to address a particular barrier. Tables 
and figures could be somewhat more clear by identifying statistical significance in footnotes, and 
indicating baseline re-measurement in intervention points. There was a clear and concise 
description of data collection cycles for each measure. Statistical significance should be indicated 
and baseline re-measurement and intervention points should also be labeled. There were well- 
described limitations and implications for these limitations for each measure. The interpretation of 
the study findings indicate that the intervention has been successful and there were 
recommendations for follow-up to implement two additional years of this program. This would 
allow for measurement of sustained improvement as well as opportunity for additional 
improvement. The baseline methodology was modified for one measure over time, but this was 
clearly described and the implications for this change were understood. There was very good 
presentation of statistical significance testing and findings. The reported improvements had face 
validity, demonstrated statistical evidence of true improvement, and demonstrated sustained 


improvement through repeated measurements over comparable time periods. 


This study has led to highly credible and valid findings and is considered a Best Practice approach to 
asthma management. This is a very well done project with a good presentation of measures and 
results. Data analysis was well described for each measurement cycle. Statistical analysis of change 


over time was conducted, and the success of the interventions was discussed. 
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In the future, identify more clearly the study questions. Outline the data analysis plan and the 


Statistical tests to be conducted. Include attachments and survey tools for future reviews and 


modify the QIA Form as needed to include components of PIPs. 


STRENGTHS 


Both PIPs validated either produced or were capable of producing highly credible findings to 
evaluate the effectiveness of care. Blue Advantage Plus demonstrated effectiveness and stability 
of intervention with the Improving Care for Asthmatics PIP. This is considered a Best Practice 
for widespread application to MC+ Members. Positive findings regarding the effectiveness of the 
Lead Testing PIP will also likely result ina Best Practice approach for MC+ Members. 


Blue Advantage Plus employed statistical significance testing in demonstrating the effectiveness of 
the intervention and sustainability of improvement in outcome measures over time. 


Blue Advantage Plus provided explicit descriptions of definitions, sources, and time frames of 
outcome measures. Numerators and denominators were well defined and could be replicated. 


Blue Advantage Plus reported of numerators and denominators as well as rates. This allows for 
trending of rates over time, consistency in measurement, and examination of raw data for the 
purpose of critically evaluating the process and variation of the interventions over time. 


There was excellent use and modification of the Quality Improvement Activity Form for the 
purpose of planning, identifying, and summarizing the findings of Performance Improvement 
Projects. The study rationale was very well written and described. 


AREAS FOR IMPROVEMENT 


There was some confusion about sampling. A sample is a subset of a target population that is 
selected for inclusion in a study, so that findings can be extrapolated to the entire population. 
This is not the same as the number or proportion of those who chose to participate or for 
whom data could be collected. This would be the response, or participation rate (the number 
and percent in the sample or population that were targeted that also participated). 


Articulating interventions, study questions, hypotheses, findings, and interpretations of findings. 
Findings and conclusions made about the interventions or barriers, as well as recommendations 
for each measurement period should be summarized. 
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RECOMMENDATIONS 


|. When submitting PIPs for review, include attachments, surveys, graphic displays, and summaries 
of the intervention(s). 


2. Specify the population, or the proportion of the MC+ population included in the study and in 
the intervention (e.g., all MC+ members; or “55% of the target population consisted of MC+ 
members; how many people and from what population/sample were contacted by telephone’). 
Ideally, all analyses and findings of the outcomes are disaggregated and reported for MC+ 
Members. For the intervention and process measures, it would be important to report at 
minimum, the number and proportion of MC+ Members included in the intervention. 


3. Report the type of statistical tests used and the degrees of freedom. 


Validation Of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Blue Advantage Plus. Blue Advantage Plus submitted the requested 
documents on November 23rd, 2004. The EQRO reviewed documentation between November 
23rd, 2004 and February 28th, 2005. On-site review time was used to conduct follow-up questions 


and provide feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Information Systems Capability Assessment (ISCA) submitted by Blue Advantage Plus 

e The Baseline Assessment Tool (BAT) submitted by Blue Advantage Plus 

e Ernst & Young’s NCQA HEDIS 2004 Compliance Audit Report 

e Letters of communication between the EQRO and Blue Advantage Plus 

e Blue Advantage Plus policies pertaining to HEDIS 2004 rate calculation and reporting 

e Blue Advantage Plus Information Services (IS) policies on disaster recovery 

e Blue Advantage Plus’s HEDIS implementation work plan and HEDIS committee agendas for 2004 
e Data warehouse validation procedures for the CRMS software 

e DB2 data warehouse models of the interim data warehouse 


The following are the data files submitted for review by the EQRO: 


e Medicaid adoles demoninator_members_with DOB.xls 
e Medicaid adoles numerators_ members with DOB..xls 
e Medicaid adoles numerators_ members with DOB.xls 

e Medicaid adoles_numerator_with_claims.xls 

e Medicaid _asthma_demoninators_with_claims.xls 

e Medicaid _asthma_demoninator_members_with DOB.xls 
e Medicaid asthma_numerators_ members with DOB.xls 
e Medicaid _asthma_numerators_with_claims.xls 
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Interviews 

The EQRO conducted on-site interviews with Darren Taylor, Barb Purdon and Marilyn Allison at 
Blue Advantage Plus in Kansas City on Tuesday, March | 5th, 2005. This group was responsible for 
calculating the HEDIS performance measures. The objective of the visit was to verify the data, 
methods, and processes behind the calculation of the three HEDIS 2004 performance measures. 
This included both manual and automatic processes of information collection, storing, analyzing, and 


reporter. 


FINDINGS 

Blue Advantage Plus used the Administrative Method for calculation of the HEDIS 2004 Adolescent 
Well-Care Visits and Use of Appropriate Medications for People with Asthma measures. The 
HEDIS 2004 Adolescent Immunization Status, Combination #1 measure was not calculated. MCO 
to MCO comparisons of the rates of HEDIS 2004 Adolescent Immunization Status Combination #1, 
Adolescent Well-Care Visits, and Use of Appropriate Medications for People with Asthma measures 
were conducted using two-tailed z-tests. For comparisons that were statistically significant at the 
95% confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 


significance levels (p < .05) are reported. 


The rate for Blue Advantage Plus for the HEDIS 2004 Adolescent Well-Care Visits measure was 
31.20%, which was comparable to the statewide rate for all MC+ MCOs (30.13%; z = .23, 95% Cr: 
26.32%, 36.08%; n.s.). The rate for Blue Advantage Plus for the 2004 HEDIS Use of Appropriate 
Medications for People with Asthma was 67.39%, which was significantly higher than the statewide 
rate for MC+ MCOs (63.92%, z = .92; 95% Cl: 63.75%, 71.03%; p > .95). 


The following sections summarize the findings of the process for validating each of the performance 
measures in accordance with the Validating Performance Measures Protocol. The findings from all 

review activities are presented according to the EQRO validation activity, with the findings for each 
measure discussed within the activities as appropriate. Please refer to the tables in the main report 


for activities, ratings, and comments related to the Attachments. 
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Data Integration and Control 

Blue Advantage Plus used a NCQA-certified vendor application from McKesson, Inc. for calculation 
of rates for the HEDIS 2004 measures. The EQRO was provided with a process overview of the 
FACETS claims management system and a validation overview of the CareEnhance Resource 
Management System (CRMS) data warehouse. The EQRO was given a demonstration of the data 
flow and integration mechanisms for external databases for these measures, and provided with a 
layout of the data structure of the internally-developed data warehouse for storing interim data. 
For the two measures calculated, Blue Advantage Plus was found to meet all criteria for producing 
complete and accurate data (see Attachment V: Data Integration and Control Findings). There 
were no biases or errors found in the manner in which Blue Advantage Plus transferred data into 
the repository used for calculating the HEDIS 2004 measures of Adolescent Well-Care Visits and 


Use of Appropriate Medications for People with Asthma. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate (see Attachment VII: Data 
and Processes Used to Calculate and Report Performance Measures). Blue Advantage Plus met 
nearly all criteria that applied for the two measures validated. The two criteria that were not met 
involved the use of statistical significance testing to document changes in performance over time and 
using this information to support claims of improvement or stability in performance over time. 
Although many auditors conduct statistical tests to examine various aspects of data validity, such 
tests should be used by the MCO to assess the significance of change related to quality 


improvement activities and operational changes. 


Processes Used to Produce Denominators 

Blue Advantage Plus met all criteria for the processes employed to produce the denominators of 
both performance measures validated (see Attachment X: Denominator Validation Findings). This 
involves the selection of eligible members for the services being measured. Denominators in the 
final data files were consistent with those reported on the DST for the two measures validated. All 
members were unique and the date of birth ranges were valid. A total of 6,593 members eligible 
were reported and validated for the Adolescent Well-Care Visits measure; and 788 members 
eligible were reported for the denominator of the Use of Appropriate Medications for People with 
Asthma measure. Age ranges, dates of enrollment, medical events, and continuous enrollment were 


programmed to include only those members who met HEDIS 2004 criteria. 
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Processes Used to Produce Numerators 

Both measures validated included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2004 criteria (see 
Attachment XIll: Numerator Validation Findings). Given that the Adolescent Immunization Status, 


Combination # | measure rate was not calculated, it was not possible to estimate the total bias. 


There were a total of 2,057 administrative hits reported and validated for the HEDIS 2004 
Adolescent Well-Care Visit measures. The dates of service and medical event codes (CPT and ICD- 
9 CM) were all within the valid ranges. The rate validated by the EQRO for Adolescent Well-Care 


Visits was 31.20% , with no observed bias. 


The HEDIS 2004 Use of Appropriate Medications for People with Asthma measure was another 
measure validated. The dates of birth and dates of service were in the valid range. The final rate 


was calculated by the EQRO was 67.39%, with no observed bias. 


Sampling Procedures for Hybrid Methods 
No sampling or medical record reviews were conducted or validated for the performance measures 
validated. Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: Sampling 


Validation Findings do not apply to the Administrative Method. 


Submission of Measures to the State 

Blue Advantage Plus submitted the DST for two of the three measures validated to the SPHA (the 
Missouri Department of Health and Senior Services: DHSS) in accordance with the Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 
Improvement Strategy. The Adolescent Immunization Status, Combination #1 measure was not 
calculated by Blue Advantage Plus and thus was not reported. The 2004 Compliance Audit Report 
indicated that the Blue Advantage Plus did not calculate the measure, but no further information was 


available regarding the reason. 
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Determination of Validation Findings and Calculation of Bias 

As referenced earlier, there was no bias found in the reporting of numerators, denominators, or 
rates of the two HEDIS 2004 performance measures validated. The Adolescent Immunization 
Status, Combination #1 measure was not calculated or reported and was thus unable to be 


validated. 


Table |. Estimate of Bias in Reporting of HEDIS 2004 Measures. 


Measure Estimate of Bias Di a-verd (ola Me) mm ja] ait-1 a=) 


Adolescent Immunization Status, Combination #1 Unable to determine Unable to determine 
Adolescent Well-Care Visits 0.00% None 
Use of Appropriate Medication for People with Asthma 0.00% None 





Final Audit Rating 
The Final Audit Rating for each of the performance measures was based on the findings from all data 
sources that were summarized in the Final Performance Measure Validation Worksheet for each 


measure. 


Table 2. Final Audit Validation Rating for Performance Measures 


Measure Final Audit Rating 
Adolescent |!mmunization Status, Combination #1 Not Valid 


Adolescent Well-Care Visits Fully Compliant 

Use of Appropriate Medications for People with Asthma Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure was substantially 
compliant with State specifications and had only minor deviations that did not significantly bias the reported rate; A significant bias in the 
rate was defined as a number calculated by the EQRO that fell outside the 95% confidence interval of the rate reported by the MCO. 
Not Valid = Measure deviated from State specifications such that the reported rate was significantly biased. This designation is also 
assigned to measures for which no rate was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. The calculation of the HEDIS 2004 Adolescent Well-Care Visits and the Use of Appropriate 
Medications for People with Asthma measures were fully compliant with SMA and SPHA 
specifications. 


2. Blue Advantage Plus used NCQA-certified software (CRMS of McKesson) for the HEDIS rate 
calculations. This application was reviewed and tested by NCQA-certified auditors and has 
been found to accurately generate rates. There is an effective and documented CRMS validation 
process in place. The data for the calculation of HEDIS measures is loaded into the CRMS 
warehouse from the internally-developed central decision support (CDS) repository. 


3. Blue Advantage Plus had documented policies and procedures on calculation of HEDIS 
measures. There was an organization-wide, high-level policy for the calculation of HEDIS 
measures. There were associated project plans for calculation of HEDIS measures and effective 
disaster recovery policies in place. 


4. There was an efficient data integration process in place. The pharmacy data was incorporated 
from Argus into the CDS repository. The CDS repository also collects data from the FACETS 
claims management system. 
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There were good system edits, validity checks and balancing for data accuracy both within the 
FACETS claims management system and the CDS repository. 


The information system had a unique business entity key for calculation of continuous 
enrollment for the Blue Advantage Plus’s members. This facilitates non-duplication of member 
data entry in the process. 


Blue Advantage Plus is currently migrating to a new data warehouse system that incorporates an 
NCQA-certified vendor for HEDIS rate calculations. 


Upon review of the preliminary of findings of the performance measure validation, Blue 
Advantage Plus acknowledged the need and expressed the intent to calculate and report the 
Adolescent Immunization Status, Combination #I measure to the SMA and SPHA. 


AREAS FOR IMPROVEMENT 


There is a need to report the Adolescent Immunization Status, Combination #I measure to the 
SPHA and SMA. Reporting of the Adolescent Immunization Status performance measure is 
mandatory under the Code of State Regulations (19-CSR §10-5.010) and is also required under 
the SMA Contract for Medicaid Managed Care (see Quality Improvement Strategy). 


Data analysis should incorporate tests of statistical significance to assess the statistical 
significance of changes in rates from year to year. There is an opportunity to include tests of 
Statistical significance to identify changes or stability from year to year. 


RECOMMENDATIONS 


Report the Adolescent Immunization Status, Combination #I rate to the SMA and the SPHA 
regardless of whether or not the measure is reported to NCQA. The rates provide a valuable 
comparison mechanism for the SMA to facilitate policy making and quality assessment. 


Conduct and document statistical comparisons on rates from year to year. 


Continue the migration to a new data warehouse system with an integrated NCQA-certified 
vendor. This would provide better analytical processes for future performance. 


Validation of Encounter Data 


FINDINGS 


The findings for the encounter data validation are organized according to the encounter data 


evaluation questions presented in the Technical Methods section for the encounter data validation in 


the aggregate report. Please refer to the main report for detailed objectives, technical methods and 


procedures for encounter data validation. 


Performance Management Solutions Group 350 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


What is the Baseline Level of Completeness, Accuracy, and 
Reasonableness of the Critical Fields? 


For the Medical claim type, there were a total of 75,956 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. 


The Outpatient Claim Type field was |100.00% complete, accurate and valid. 

The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

The Outpatient First Date of Service field was 100.00% complete, accurate and valid. 
The Outpatient Last Date of Service field was 100.00% complete, accurate and valid. 
The Outpatient Units of Service field was |00.00% complete, accurate and valid. 
The Outpatient Procedure Code field was 100.00% complete, accurate and valid. 
The Outpatient Place of Service field was |00.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


eS a a CY 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 46.74% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 20.27% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
9.54% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. 


For the Dental claim type, there were a total of 7,491 encounter claims paid by the SMA for the 
period January |, 2004 through March 31, 2004. All of the fields examined were |00.00% complete, 


accurate and valid. 


For the Home Health claim type, there were a total of 567 encounter claims paid by the SMA for 
the period January |, 2004 through March 31, 2004. All of the fields except the second through fifth 
Diagnosis Code fields examined were 100.00% complete, accurate and valid. The second, third, 
fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for completeness, 
accuracy, and validity established by the SMA (54.67%, 19.75%, 6.88%, 3.35%, respectively). The 


remaining fields were blank (incomplete, inaccurate, and invalid). 
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For the Inpatient claim type, there were a total of |,569 encounter claims paid by the SMA for the 


period January |, 2004 through March 31, 2004. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 


The Admission Type field was 00.00% complete, accurate and valid. 


= aw NM 


The Admission Date field was 100.00% complete and accurate, and 94.26% valid. There were 
90 invalid dates ranging from 09/13/2003 — 12/31/2003. 


5. The Discharge Date field was 100.00% complete with the correct number of 99.43% (with 9 
entries of “99999999"). Valid values were present 96.75% of the time. In addition to the nine 
invalid “99999999” entries, 42 invalid dates ranged from 04/01/2004 — 04/29/2004. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


— at 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (84.51%, 68.36%, 48.50%, 36.90%, 
respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 94.26% valid. There 
were 90 invalid dates of service ranging from 09/13/2003 — 12/31/2003. 


l1. The Last Date of Service field was 100.00% complete and accurate, and 97.20% valid. There 
were 44 invalid dates of service ranging from 04/01/2004 — 04/29/2004. 


|2. The Revenue Code field was 100.00% complete, accurate and valid. 


13. The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were a total of 33,155 encounter claims paid by the 


SMA for the period January |, 2004 through March 31, 2004. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete and accurate, and valid. 
The Last Date of Service field was 100.00% complete and accurate, and valid. 


The Units of Service field was 100.00% complete, accurate and valid. 


So Mv S| YN 


The Outpatient Procedure Code field was 100.00% complete and accurate, and 100.00% valid 
(with rounding). The invalid code was a ““VWW0039” entry. 


7. The Outpatient Revenue Code field was 100.00% complete, accurate and valid. 


8. The first Diagnosis Code field was |00.00% complete, accurate and valid. 
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9. Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 57.39% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis Code field 
was 2659% complete, accurate and valid (incomplete, inaccurate, and invalid). The remaining 
fields were blank (incomplete, inaccurate, and invalid). The fourth Diagnosis Code field was 
13.78% complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, 
and invalid). The fifth Diagnosis Code field was 7.66% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). 


10. For the Pharmacy claim type, there were a total of 52,577 claims paid by the SMA for the period 
January |, 2004 through March 31, 2004. All fields examined were 100.00% complete, accurate 
and valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Blue Advantage Plus, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. Medical, 
Dental, Home Health, and Pharmacy claim type critical fields examined were |00.00% complete, 
accurate, and valid. The Discharge Date for the Inpatient claim type contained some invalid codes, 


and the Outpatient Procedure Code field contained some invalid procedure codes. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Blue Advantage Plus 
demonstrated rates consistent with the average for all MC+ MCOs for the Medical, Dental, 
Outpatient Hospital, and Pharmacy claim types; and a significantly higher rate for Home Health 
encounter claim types. There was a significantly lower rate of Inpatient encounter claim types for 
Blue Advantage Plus than the average for all MC+ MCOs. The relation between the higher Home 
Health and lower Inpatient encounter claims may be related to claims administration processes. 
These findings suggest high levels of complete data and at least moderate access to care for Blue 
Advantage Plus members. The findings from the performance measure analysis indicate that Blue 
Advantage Plus found significantly higher administrative hits for the HEDIS 2004 Adolescent Well- 
Care Visits and Use of Appropriate Medications for People with Asthma measures than the average 


for all MC+ MCOs. This also suggests high levels of complete data. 
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To what Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim types for 
the period of January |, 2004 through March 31, 2004 for medical record review. Of the 95,566 
Medical encounter claim types in the SMA extract file for January |, 2004 through March 31, 2004, a 
total of 100 encounters were randomly selected. This was an oversample, as the minimum required 
sample size was 88 per MC+ MCO. Providers were requested to submit both 

medical records and claim forms or claim histories for review. There were 84 medical records 
(84.0%) and I|7 claim forms (17.0%) submitted for review. Encounters for which no documentation 
was submitted were unable to be validated. The match rate for procedures was 63.0%, with a fault 


rate of 37.0%. The match rate for diagnoses was 57.0%, with a fault rate of 41.0%. 


What Types of Errors were Noted? 

An error analysis of the errors found on the medical record review and claim forms for procedure 
and diagnosis codes and descriptors was conducted. For the diagnosis code in the medical record, 
the reasons for diagnosis codes not matching the SMA extract file were missing information (n = 85) 
or incorrect information (n = 3). For the diagnosis description in the medical record, the reasons 
for diagnoses not matching the SMA extract file were missing or illegible information (n = 41), or no 
match with the description of the symptoms based on the information in the medical record (n = 

| 1). For the diagnosis code on the claim form, the reason for diagnosis codes not matching the 
SMA extract file was missing or illegible information (n = 85). The reason for the diagnosis 
descriptor on the claim form not matching the SMA extract file included missing or illegible 


information (n = 79). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA extract 
file not being supported by documentation in the medical record were missing information (n = 6), 
downcoding (billing a service that is reimbursed at a lower rate or for less time than actually spent 
with the patient of the information in the medical record; n = 2), incorrect codes (n = 6), upcoding 
(billing a service that is reimbursed at a higher rate or for more time than actually spent with the 
patient; n = 7), illegible information (n = |), and not enough information to code (n = |). For the 


procedure code on the claim form, the reason for procedure codes not matching the SMA extract 
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file was missing or illegible information (n = 86). For the procedure description on the claim form, 
the reason for procedure codes not matching the SMA extract file was missing or illegible 


information (n = 92). 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to the 
SMA encounter claim extract file due to the variability across MC+ MCOs in the submission of data 
for the encounter data validation. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in national 
standard format for the MC+ Managed Care Members represented in the encounter claim sample 


selected by the EQRO for validation. 


For purposes of the EQRO, Blue Advantage Plus had no difficulty submitting files for paid claims in 
the requested file layouts encounter data validation processes. Files were requested in flat file 
(comma-delimited), machine readable format. The following section summarizes the findings for 


reviewing the files submitted for encounter data validation analysis. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


The documentation received from DMS (dated December 28, 2004) from Mary Lyn Childs and Judy 
Reynolds indicates that “Drug claims are in the old format prior to October 2004, and the physician, 
hospital, and dental claims are in the new format that was implemented October, 2004. Your 
request includes the submission of data in the format used prior to October 2004. As you are 
aware, the State continues to provide direction to the Plans as to how to submit encounter data in 
accordance with the Encounter Data Corrective Action Plan. According to State guidelines, we 
have converted our data submission format and are unable to go back to the previous format. For 
this reason, the data you are requesting can only be submitted in the current format. We are 
unable to submit denied claims per your request. We do not send rejected encounters to the State. 
Dental denied claims are on a totally different table from paid encounters. Accessing this 
information would require requesting a new IS project. We are checking to find out what format the 


drug encounter information will be available from our PBM.” 


Performance Management Solutions Group 355 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouR!I MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


bhc_Dental_encs.txt 136KB Text Document 12/16/2004 10:08 AM 
This file was in NSP/CMS 1500 file layout and was able to be loaded for analysis. 


2. bhc_Drug_encs.txt 372KB Text Document 12/16/2004 10:18 AM 
A total of 476 Pharmacy claim types were able to be loaded for analysis. 

3: bhc_Hosp_encs.txt 237KB Text Document 12/16/2004 10:09 AM 
This file was in UB-92 file layout and was able to be loaded for analysis. 

4. bhc_Phys_encs.txt 1.279KB Text Document 12/16/2004 10:11 AM 
This file was in NSF/CMS 1500 file layout and was able to be loaded for analysis. 

5. EQRO-NonPaid_Claims_2004.xls 
This file was unable to be loaded for analysis. 

STRENGTHS 

|. The critical field validation of all six claim types resulted in few fields under the SMA established 
threshold of 100.00% accuracy, completeness, and validity. 

2. The critical fields evaluated for the Medical, Dental, Home Health and Pharmacy claim types 
were 100.00% complete, accurate, and valid. 

3. The match between medical records and the SMA encounter claims database for procedure 
codes and descriptions was significantly higher than the average for all MC+ MCOs; and the 
match rate between diagnosis codes and descriptions was consistent with the average for all 
MC+ MCOs. Errors were related to missing or illegible information, and no upcoding or 
downcoding errors were found. 

4. The rate of claims for each claim type was consistent with the average for all MC+ MCOs, 


except for the rate of the Pharmacy claim type, which was higher and the rate of Inpatient claim 
type which was lower than the average for all MC+ MCOs. The rates of identification of 
administrative claims for adolescent well care and medications for the performance measures 
were higher than the average for all MC+ MCOs. The rate of claims suggest at least moderate 
levels of completion of data and access to services. 


AREAS FOR IMPROVEMENT 


For the Inpatient claim type, there were invalid dates in the Discharge Date field. 


2. The Outpatient Procedure Code field in the Outpatient Hospital claim type contained | invalid 


entry. 
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RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the UB-92 file 
layout for the Outpatient Procedure Code and Discharge Date fields, and run validity checks 
after the programming of new edits. 


MCO Compliance with Managed Care Regulations 


METHODS 

Objectives, technical methods, and procedures are described earlier in this report. This section 
describes the documents, data, and persons interviewed for the Monitoring Medicaid Managed Care 
Organizations (MCOs) protocol for Blue Advantage Plus (BA+). The EQRO reviewed 
documentation between December |, 2004 and February 28, 2005. On-site review time was used 
to conduct follow-up questions, review additional documentation made available by Blue Advantage 
Plus, and provide feedback and recommendations regarding compliance with federal Medicaid 


Managed Care Regulations. 


Document Review 
In addition to the documents previously identified that were reviewed at each MC+ MCO, Blue 
Advantage Plus was requested to provide the following documents on-site: 


e Member Handbook 

e Provider Handbook 

e Provider Agreements (Physician’s Participation Agreements) 

e Samples of both Grievances and Appeals related to members and providers 


Additional documentation provided by Blue Advantage Plus on-site: 


e 2004 Blue Advantage Plus Telephone Follow-Up Prevention Program report 
e Office Site Assessment Standards 2005 

e Medical Record Documentation Review Standards 2004 

e Provider Eligibility policy 

e Physician Access Standards policy 

e Blue Advantage Plus Care Coordination policy 

e Blue Advantage Plus Specialist as Primary Care Physician policy 

e Blue Advantage Plus Analysis of Vendor and Downstream Practitioner Contracts 
e Blue Advantage Plus Hospital Quality Initiative report 

e Provider Directory 

e §=6 Ancillary Provider Agreement 

e Processing and Payment of Claims by BC/BSKC 
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New Directions Behavioral Health supplied the following documents regarding their services to Blue 


Advantage Plus: 


e 2004 Blue Advantage Plus Member Satisfaction Survey results 


e Personal Transitions Services Assessment tool 


e Blue Advantage Plus Acute Care 7 Day Follow-Up report 


e New Directions Physicians Helpline tools 


e Personal Transition Services brochures 


Interviews 


The following individuals were interviewed during the on-site review: 


Plan Administration 

Judy Brennan, Director of State Programs, Blue 
Advantage Plus Plan Administrator 

Dr. Loretta Britton, Vice President and Medical 
Director 

Shelly Bowen, AVP, Quality Management 

Mary Lyn Childs, Special Programs Coordinator 


Quality Assurance 

Judy Brennan, Director of State Programs 
Shelly Bowen, AVP, Quality Management 
Cheryl Banks, Manager, Quality Performance 
Measurement 

David Wilson, Director, Population Management 


Member Services/ Case Management 

/ Utilization Management 

Judy Brennan, Director of State Programs 

Mary Lyn Childs, Special Programs Coordinator 
Casey Cantin, Director, Local/National Claims 
Gene Payne, Supervisor, Blue Advantage Plus 
Customer Service 

Sandy Wederguist, Director, Medical Management 
- Case Management 

Rhonda J anky, Director, Medical Management - 
Prior Authorization 

Carol Kemper, Senior Director, Quality 
Management, Children’s Mercy Hospital 


FINDINGS 


Enrollee Rights and Protections 


Provider Services 

Wayne Bunge, Vice President, Provider Services 
Randy Meyer, Director, Hospital Services 
Dennis Radio, Director, Professional Services 
Derrick Swetlishoff, Manager, Network 
Development 

Judy Brennan, Director of State Programs 

Mary Lyn Childs, Special Programs Coordinator 


Mental Health 

Myron Unruh, AVP, Clinical Operations 

Garth Smith, Director, Network Operations 

Lisa Woodring, Director, Prevention and Support 
Services 

Judy Brennan, Director of State Programs 


Blue Advantage Plus exhibited a committed and enthusiastic approach to ensuring member rights 


and protections. The MCO used every possible opportunity to make initial contact with members 


to ensure that they chose a PCP, and had all introductory information including the Member 


Handbook. They used these initial contacts to ensure that members had written information in a 
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language and format that they understood. The MCO used a TDD line with members who were 
hearing-impaired. They could produce member information in Braille if requested, and were willing 
to spend the time reading or explaining benefits and policies to members on the telephone if 


required. 


Blue Advantage Plus believed its members were an important component of the larger commercial 
organization and they treated members with respect and dignity. Staff attempted to provide good 
service to members. They made a concerted effort to ensure that their healthcare needs were met. 
Blue Advantage Plus knew that members had a right to all services in their benefit package. The 


MCO saw its mission as eliminating barriers to members obtaining needed healthcare services. 


Ratings regarding compliance with Enrollee Rights and Protections regulations (84.6%) reflected two 
policy issues to be resolved for the MCO to achieve full compliance. The MCO must complete 


these policies and submit them to the SMA for their final review and approval. 


Quality Assessment and Performance Improvement 

Access Standards 

Blue Advantage Plus had an extensive provider network in place. In addition to the Blue Advantage 
Plus network, they had additional providers, particularly specialists, who were part of the larger Blue 
Cross/Blue Shield of Kansas City (BC/BSKC) network, that agreed to see Blue Advantage Plus 
members if requested. The MCO did not object to utilizing out-of-network providers. They 
expressed a willingness to do this any time this was in the best interest of the member. Blue 
Advantage Plus responded to every member complaint regarding provider access, which assisted the 
MCO in evaluating network adequacy. The MCO monitored access standards regarding 
appointment and waiting room time using a number of methods. They conducted on-site office 
visits, Consumer surveys, and responded to consumer complaints. The MCO used the information 
gained to initiate corrective action, but stated that occurred very rarely. 

Blue Advantage Plus operated an in-depth case management program. Case managers viewed their 
goal was to assist members in receiving all the healthcare services required. Case managers actively 
attempted to identify members with special healthcare needs. They provided assessments to all 


members who were referred for or requested special services. The MCO staff indicated that if they 
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learned that a child had an illness requiring any specialized care, they categorized them as having 
special healthcare needs to ensure that they received additional services or assistance they might 
require. Case management staff involved the member and PCP in defining an appropriate treatment 


plan and worked to ensure that members obtained the services identified in that plan. 


Ratings regarding Access Standards regulations (82.4%) reflect that Blue Advantage Plus hada 
number outstanding policies to complete to be fully compliant with these regulations. This included 
correcting language in the subcontractor agreement that was considered non-compliant by the SMA, 
and completion of policy regarding coordination of care between PCPs and specialists, including 


expected response time for consultations. 


One additional concern existed regarding the Member Handbook. On page 1|4, under services 
requiring prior authorization, Prenatal Care was listed. This was not a listed service in other MC+ 
MCO handbooks. This discrepancy was noted as it may discourage members who are pregnant, or 


who believe they are pregnant from seeking appropriate healthcare. 


Structure and Operation Standards 

Ratings for compliance with Structure and Operation Standards regulations (90%) reflect that Blue 
Advantage Plus was substantially compliant with these regulations. They had credentialing and 
recredentialing standards that exceeded the requirements. Blue Advantage Plus used a twelve- 
member review panel comprised of providers form the community who reviewed potential network 
members, and re-reviewed existing providers for the recredentialing process. They had stringent 


policy for recognizing and eliminating providers who were no longer licensed. 


All policy and practice was in place regarding the disenrollment process. Language in the Member 


Handbook explained the steps necessary for a member to change MC+ MCOs. 


The area of concern related Structure and Operation Standards was the issue of approved 
subcontractor agreements. The SMA disallowed certain language and at the time of the on-site 
review the revisions had not been submitted by the MCO for final review and approval. This issue 


must be resolved for Blue Advantage Plus to be fully compliant. 
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Measurement and Improvement 

Blue Advantage Plus was involved in the community-based Kansas City Quality Improvement 
Consortium. This group developed clinical practice guidelines for diabetes, asthma, and was in the 
process of developing obesity guidelines. The MCO used additional national standards, particularly 
in the area of behavioral healthcare. These guidelines were updated every two years and more 
often if nationally recognized information changes. The MCO believed the use of these standards 
improved the quality and consistency of care for members. Practice guidelines were disseminated 
through initial and on-going provider training, inclusion in the Providers Office Guide, and MCO 
newsletters. Application of guidelines was reviewed through the normal monitoring processes. 
Provider profiles included monitoring results on the use of clinical guidelines. The Medical Director 


became involved if a question about a standard of care was raised by a member or MCO staff. 


Blue Advantage Plus was involved in the BC/BSKC Corporate Quality Program. The MCO believed 
this had a significant benefit for members. Blue Advantage Plus demonstrated software during the 
on-site review that assisted in indicating trends in healthcare needs. The development and 
utilization of that software that created predictive modeling was the result of issues discussed in that 


committee and became a quality initiative. 


The MCO did submit two Performance Improvement Projects for validation. These PIPs produced 
credible findings and sustainability of statistically significant improvement over time. The full report 
regarding PIPs can be found in the appropriate section of this report. The MCO explained that they 
did not compile or submit findings on one of the required performance measures. Details of the 
Validation of Performance Measures can be found in the appropriate section of this report. The 
MCO did have a health information system (HIS) that was capable of meeting the MC+ Managed 
Care program requirements. Encounter Validation Data was not submitted in a format that allowed 


analysis as required. 


Ratings for compliance with the Measurement and Improvement regulations (72.7%) reflect the need 


to improve data submission. 
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Grievance Systems 

Ratings for compliance with Grievance Systems regulations (100%) indicate that the MCO had 
completed all requirements regarding policy and practice in their grievance and appeal system. Files 
for both grievances and appeals, filed by members and providers, were reviewed on-site. All files 
followed prescribed policy and timelines. Notices to members were sent within required 
timeframes and contained all required information. This information included the member’s ability 
to file a State Fair Hearing simultaneously with an MCO appeal, or later if the outcome of an appeal 
was not favorable to the member. All policy and information to members included the message that 
members can maintain healthcare coverage while an appeal was pending, and explained the 
member’s responsibility to pay for charges if a decision was reached to deny the disputed service. 
The Medical Director was involved in many disputed issues and made every attempt to ensure that 
members obtained the healthcare needed. If an appeal was filed, it was recorded in the MCO 
tracking system to ensure that another qualified individual reviewed the information submitted to 
obtain an independent appeal decision. The MCO took the grievance and appeal system very 


seriously and used information generated to inform their quality improvement process. 


Summary and Follow-up 

Behavioral Health 

During the past year, New Directions Behavioral Health (NDBH) became a wholly owned subsidiary 
of Blue Cross/Blue Shield of Kansas City. The MCO reported that this change increased the 
availability of information sharing between behavioral health providers and PCPs. New Directions 
Behavioral Health case managers co-case managed with Blue Advantage Plus staff when members 
received both behavioral health and physical health services. NDBH case managers were informed 
when members they served were hospitalized for any reason. All Emergency Room visits that 


appeared to be the result of mental health needs were reviewed by both systems. 


Blue Advantage Plus was an active member in committees and activities pertinent to the local 
healthcare community. The MCO believed that this enabled them to provide the best services 


available to their members. 


If a member was admitted to a hospital as the result of mental health problem, the PCP was notified. 
There was follow-up after release to ensure that the member kept appointments. In 2003 45% of 


members kept after-care appointments. In 2004, 60% were keeping aftercare appointments. The 
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NDBH information system reminded staff to conduct follow-up contacts on a regular basis. An 
acuity system was in place that indicated the frequency of contacts discharged patients should 
receive. NDBH placed over 5,139 follow-up calls in 2004. They were successful in reaching 


members 61% of the time. Contacts were attempted for all discharged members. 


Blue Advantage Plus and NDBH continued to collaborate with Gillis Center to provide in-home 
crisis intervention. Intensive services included assessment, parenting skills training, and school 
coordination. The Gillis Center staff assisted the member and/or the family in connecting to a 
therapist for ongoing support at the appropriate CMHC if they required continued wrap-around 
services, or through NDBH if therapy services were recommended. Blue Advantage Plus and 
NDBH believed that this resource assisted families in preventing a crisis that created a need for or 


caused longer inpatient treatment, or led to more serious external interventions for the family. 


Access to Care 

There were a number of follow-up issues regarding different aspects of access to care that Blue 
Advantage Plus has resolved. Blue Advantage Plus utilized additional incentive payments for 
physicians who accepted and treated MC+ members. Blue Advantage Plus used BC/BSKC 
commercial network specialists who agreed to see MC+ members, but were reimbursed at the 
commercial network rate. The MCO used out-of-network physicians when necessary. Blue 
Advantage Plus developed specific policy on access standards that they described as exceeding the 


requirements of their MC+ Medicaid Managed Care contract. 


Collaborative Projects 

Blue Advantage Plus continued to be actively involved in understanding and serving foster children 
who were part of the federal Consent Decree in Jackson County. The MCO was involved in 
ensuring that all aspects of case management, as required by the Consent Decree, were in place, 
managed, tracked and recorded to meet required standards. Attention to this important group of 


members was noteworthy. 


Blue Advantage Plus was an active member in committees and activities pertinent to the local 
healthcare community. The MCO believed that this enabled them to provide the best services 


available to their members. 
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Blue Advantage Plus was working with the Local Investment Commission (LINC) to bring a 


community-based healthcare clinic to the McCoy School District. 


Correct Contact Information 


The MCO completed a formal study on the number of members who came into the MCO with 


incorrect contact information. Blue Advantage Plus asserted that the study determined that the time 


and resources expended on correcting addresses and telephone numbers was a serious problem, 


which the MCO has not been able to resolve. 


Case Management 


Blue Advantage Plus recently acquired a software package for predictive modeling analysis. This 


software was used to identify the members most likely to need case management services. The 


system was based on in-patient stay probability. It was being piloted with a subset of numbers to 


evaluate its use in the prevention of problems through case management. The focus was on 


prevention. However, the MCO believed the system could also be used in disease management. 


STRENGTHS 


Blue Advantage Plus continued its partnership with New Directions Behavioral Health and Gillis 
Children’s Center to provide intensive in-home services to families in crisis. This effort 
produced positive results for members and introduced them to community resources and 
healthcare services in an effort to teach proactive coping skills. The MCO and BHO provided 
many positive outcomes for members and their families. 


Blue Advantage Plus staff gave examples of how the MCO went beyond what is prescribed in 
the MC+ Medicaid Managed Care contract with the State to ensure that families obtain services 
that meet their health and mental health needs. The MCO demonstrated a firm commitment to 
ensuring that members’ lives were enhanced by their involvement with Blue Advantage Plus.. 


Blue Advantage Plus was starting to use a predictive modeling tool to provide information, 
analysis, and trends that would assist them in developing a proactive approach to designing 
future member services. 


Blue Advantage Plus staff throughout the organization had a thorough knowledge of programs 
and initiatives in the local healthcare community in this MC+ Region. They used this knowledge 
to ensure that members obtained all of the best services available. 


The staff at Blue Advantage Plus was well organized and exhibited strong working relationships. 
These relationships were indicative of an approach focusing on providing the best care available 
to their members and an effective utilization of resources. 
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AREAS FOR IMPROVEMENT 


|. Blue Advantage Plus should ensure timely completion of the contract compliance process so all 
policies and procedures are submitted to and approved by the SMA. This will create full 
compliance with MC+ Medicaid Managed Care contract requirements and federal regulations. 


RECOMMENDATIONS 


|. Continue MCO development in the area of utilizing of available data and member information to 
drive change and create opportunities for organizational growth and development. 


2. Continue involvement in community initiatives to open up opportunities for members. 


3. Continue to share information with NDBH, particularly in the area of hospitalization and 
prescription drugs, to ensure that members receive adequate follow-up care, and appropriate 
medications. 


4. Continue the Blue Advantage Plus/NDBH program with Gillis Center. Conduct an outcome 
study to support the positive impact of this service. 


5. Maintain partnerships with both public and private agencies in an effort to continue service 
improvements and expansions for MC+ members where this is possible. 


6. Share objective study information with the SMA and request their partnership to influence 
effective problem solving regarding access to member contact information. This is a common 
concern throughout all of the MCOs. Having well researched statistics to share may increase 
the opportunity for effective communication. 


7. Continue to define Impact Pro System usage for the benefit of Blue Advantage Plus members. 
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APPENDICES 


Appendix I: CY 2004 External Quality Review Process 
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EQRO Performs compliance review including , : : Selle eanetine cvelelien _ ; 

Information System Capabilities Assessment Evaluation Onsite review to be performed during report of administrative data EQRO drafts administrative EQRO delivers draft report 

and Analysis November 04 - January 2005. (ISCA, Encounter Data report to DMS by 4/05. 
transmission, compliance) 












































































EQRO abstracts medical 
record data and performs 
encounter data validation at EQRO evaluates data, 
each MCO site during prepares draft report. 
November - December. 





EQRO delivers draft report 
to DMS 4/05. 
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Appendix 2: Preparation of MC+ MCO 
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2004 External Quality Review Orientation 


Orientation Agenda 


-———__—_———_—_7l—_ Sa 
Introductions 


Orientation to Technical Methods and 
Objectives of Protocols 
Review of Information, Data Requests, and 
Timeframes 
Performance Measures 
Performance Improvement Projects 
Encounter Data Validation 
Compliance and Site Visits 
Closing Comments, Questions 


6/1/2005 





Materials Provided 
> 


Objectives and Technical Methods 
Validation of Performance Measures 
Validation of Encounter Data 
Validation of Performance Improvement Projects 
MCO Compliance 
Requests for information and data 
List of BHC contacts for each protocol 


Presentation 


6/1/2005 





SEE 
2004 External Quality 
Review for the Missouri 
MC+ Managed Care 
Program 


Behavioral Health Concepts, Inc. 


Performance Management Solutions Group 
Janet S. Reed, PhD, MHA 
Vice President 


6/1/2005 


Overview 
jig 
Protocol Activities 


Information and Data Requests 


Contact Persons 


6/1/2005 














Validation of 
Performance Measures 


OO 


HEDIS 2004 Measure Validation for MC+ 
Adolescent Immunization Status 
Adolescent Well Visits 


Use of Appropriate Medications for 
People with Asthma 


Administrative 
Hybrid method 


Review up to 30 medical records per 
measure sampled randomly 


6/1/2005 


Validation of 
Encounter Data 


State encounter claim database 


Randomly selected encounters from medical 
claims, with service dates January 1, 2004 — 
March 31, 2004 


Review medical records for matching claims 


Match state and MCO claims databases for 
all encounters 


6/1/2005 


6/1/2005 





Submission Requirements 
for PM Validation 


For each of the three measures: 
2 


004 HEDIS Audit Report 
Baseline Assessment Tool for HEDIS 2004 
BHC EQRO Performance Measure Checklist (Method for Calculating HEDIS Measures; Table 1.xls) 
List of cases for denominator with all HEDIS 2004 data elements specified in the measures 
Comma delimited format 
Listing of fields names and descriptions of fields (i.e., data dictionary) 
List of cases for numerators with all HEDIS 2004 data elements specified in the measures 
omma delimited format 
Listing of fields names and descriptions of fields (i.e., data dictionary) 
edges of cases det which medical records were reviewed, with all HEDIS 2004 data elements specified in 
e€ measure 


BHC will request MCOs to gather up to 30 records per measure, based on a random sample, and MCO 
will send copies 


Sample medical record ae ri for hybrid methods for HEDIS 2004 measures and instructions. 


aye befalls heets, memos, s, documentation, policies and communications within the MCO and with 
DIS auditors regarding: the reeloulaaion of the selected measures 

Policies procedures, data and information used to produce numerators and denominators 

Policies, procedures, data used to implement sampling 

Policies and procedures for mapping non-standard codes 

Others as needed 


Purpose and 
Objectives 


> 
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To obtain a baseline of the State encounter 
claim database quality (completeness, 
accuracy, and reasonableness). 


To validate the State encounter claims 
(paid) data against medical record 
documentation and obtain a baseline fault 
rate. 


To examine the match between MCO claims 


(paid) and the State encounter claims 
database. 





Sampling 


OO 


1. All State Encounter 3. All MCO encounter 
Claims, claims, 
January 1,2004 -— March31, January 1,2004  —- March 
2004 31, 2004 
(N = 100 cases per MCO) 


2. State Medical 
Encounter 
Claims 
(N = 100 per 
MCO) 
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Analyses: 2 


BHC will abstract the medical records and claims history/forms for 
each patient for the medical service provided during the entire time 
frame, enter into a database, and determine the rate(s) of matches, 
omissions and commissions between the medical record and the 
State encounter claims for each MCO. Matches will be cases that 
are consistent on patient DCN, date of service, and diagnosis or 
procedure code. 


State Medical 


Encounter Claims ‘ 
, Medical Records 
_ DCN-DOS-Code 
sree ue — wae ne 
(N = 100 per MCO) 
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Analyses: | 


OO 


Critical fields will be examined for 
completeness (data in field), accuracy 
(correct type and length of data), and 
reasonableness (valid data for field) for each 
MCO. This will be conducted for all 
encounters in the specified time frame. 


6/1/2005 


Analyses: 3 


BHC will determine the rate(s) of matches, omissions 
and errors between the State encounter claims and 
MCO encounter claims for each MCO for the sample of 
selected cases. 


MCO Encounter 
All State Encounter 


Claims (paid and 
Claims, January 1, unpaid), 
2004 — March 31, DCN-DOS-Code January 1, 2004 — 
2004 March 31, 2004 
(N = 100 per MCO) (N = 100 per MCO) 
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Encounter Data 
Validation Submission 


File 1: Provider mailing address and contact information for 
sampled claims (service dates January 1, 2004 to March 31, 
2004). This will be used for medical record requests and 
validation of the State medical encounter claims database 
against the medical record. 

File 2: All other encounters for the selected sample from 
January 1, 2004 to March 31, 2004 for selected cases, with 
provider information, from all databases (e.g., inpatient, 
outpatient, medical, dental, home health, pharmacy). This 
should include only paid claims and should be the same layout 
as that submitted to Verizon. 

File 3: All other encounters for the selected sample from 
January 1, 2004 to March 31, 2004 for selected cases, with 
provider information, from all databases (e.g., inpatient, 
outpatient, medical, dental, home health, pharmacy). This 
should include only unpaid claims and should be the same 
layout as that submitted to Verizon. 
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Validation of Performance 
Improvement Projects and 


Submission Requirements 


PIP Checklist Elements 
Project narratives, baseline measures, methods, interventions, and planned analyses. 
Examples of information are contained in the CMS protocol, Validation of Performance 
Measures[1] 
Phase-in/timeframe for each phase of each PIP[1] 
Problem identification 
Hypotheses 
Evaluation Questions 
Description of intervention(s) 
Methods of sampling, measurement 
Planned analyses 
Sample tools, measures, surveys, etc. 
Baseline data source and data 
Cover letter with clarifying information 
Raw data files (if applicable, on-site) 
Medical records or other original data sources (if applicable, on-site) 
Additional data as needed 


[1] U.S. Department of Health and Human Services, Centers for Medicare & Medicaid 
Services (2002) VALIDATING PERFORMANCE IMPROVEMENT PROJECTS A protocol 
for use in Conducting Medicaid External Quality Review Activities: Final Protocol Version 
1.0 May 1, 2 
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Validation of Performance 


Improvement Projects 
> 


Two Performance Improvement 


Projects underway in 2004 


6/1/2005 





Medical Record 
Reviews 


Encounter 
Encounter sample provided to MCO 
MCO to develop Files 1, 2, 3 (2 weeks) 


BHC to request records from providers 

directly; interim feedback to MCO (4 weeks) 
HEDIS 

Medical record samples requested from MCOs 

for 2 possible hybrid measures (N < 30 per 

measure; 4 weeks) 


6/1/2005 














Medical Record 
Reviews (Cont’d 


Request to providers 
Letter from Sandra Levels 
Instructions for submitting records 


Encounter claim supporting information, dates, 
notes, claims information 


Explanation of Confidentiality, storage of files 


Explanation of HIPAA, Business Associate 
Agreement, Health Oversight Authority 


6/1/2005 


MCO Compliance 


+ - _$§$§_$§$§$§ as 


Enrollee Rights 
Grievances and Appeals 


Quality Improvement 


Submission Requirements TBD 


6/1/2005 


Medical Record 
Reviews (Cont’d 
Reviewed and abstracted by experienced and 
certified medical coders 
Standard abstraction tools 


Matching DCN, Date of Service, Diagnosis 
Code, Procedure Code 


6/1/2005 





Site Visits 
oa 
Target for February, 2005 
MCO Compliance Reviews 
On-site activities 
Information Systems Capability Assessments 
Performance Measure Validation 
Performance Improvement Project Validation 
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Final Repo rt BHC Team and Coordination 


mr 
MCO to MCO Comparisons: oe aie 


2716 Forum Blvd., Suite 4a 


Encounter data match/fault rates for eee 


Protocol/ Activity Tel. 573-446-1816 MCO Contact 


cumn net Sine penenen er 
(HEDIS 2004) ageana@bhcinfo.com 








Performance Measure audit findings soaacs i aia Gna 
and rates Improvement Projects zr chmsaliehe ts go 
Performance Improvement Project ie 


e | e mM e nt CO mM p | a Nn ce eet Project Director 


MCO Compliance ireed@bhcinfo.com 


MCO Compliance br. Janet Reed 
Site Visits jreed@bhcinfo.com 
Research Associate 
edical Records mchumak@bhcinfo.com 
6/1/2005 G 00 

















Behavioral Health Concepts, Inc. 
Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573)446-0405 
(573)446-1816 (fax) 
(866)463-6242 (toll-free) 
www.bhcinfo.com 








Monday, November 08, 2004 
Re: 2004 External Quality Review of the MC+ Managed Care Program 


This letter represents the first request for information and data for the 2004 External 
Quality Review of MC+ Managed Care Organizations, conducted by Behavioral Health 
Concepts, Inc (BHC). We appreciate very much you and your staff's participation in the 
2004 EQRO conference calls and presentations at the All-Plan and QA & | Advisory Group 
meetings in October, 2004. We have received many good suggestions and have modified 
our procedures as needed to ensure better communication of expectations. As you know, 
we are implementing four CMS protocols: 


e Validating Performance Measures (Adolescent Well-Care Visits, Adolescent 
Immunizations, and Appropriate Medications for People with Asthma) 

e Validating Performance Improvement Projects (two State-selected PIPs per MCO) 

e Validating Encounter Data 

e MCO Compliance 


We are continuing to work with the Division of Medical Services to identify readily available 
documents and data that have been submitted to other entities in the past. For the performance 
measures, the DMS is working to obtain the File Content referenced in the legislation (19 CSR 
10-5.010) for the HEDIS 2004 measures for the MC+ product and regions (HEDIS Data 
Submission Tool) under the Cooperative Agreement between The Missouri Department of Social 
Services, Division of Medical Services and The Missouri Department of Health and Senior Services 
for Quality Assessment and Improvement. However, if this is not available from the DHSS, BHC 
will request this from MCOs by November 23, 2004. 


We have been asked by MCOs to communicate both with MC+ Plan Administrators and 
Quality Improvement Managers. Therefore, as with last year, we are sending all initial 
requests for information to the QI Managers with carbon copies of communications to the 
MC+ Plan Administrators. We will assume that MCO Managers and staff are 
communicating internally with one another regarding the External Quality Review. We 
have asked for the names and contact information of individuals at each MCO for us to 
communicate with regarding follow-up questions for each protocol. Similarly, individual 
BHC staff is assigned to implement each protocol, respond to questions, and conduct 
follow-up. Names and contact information were provided in the orientation and are 
provided again in the enclosed summary sheet. 


Enclosed are instructions for requests, mailing addresses, due dates, and contact 
information for our staff implementing each protocol. Please review them carefully, as there 
have been some minor modifications made as a result of the conference calls. This request 
replaces the items discussed in the teleconference calls. We have included a binder, tabs, 


and mailing label for your convenience. We anticipate sending requests for encounter data 
files and hybrid performance measure medical records by the end of November, 2004; 
sending requests for on-site information by the end of January, 2005; and arranging site 
visits toward the end of January 2005, to occur in February and March, 2005. 


Specific information about the implementation of the protocols can be found in the 
documents previously forwarded to MCOs for the EQRO orientation and in the 
corresponding CMS Protocols for External Quality Review. Behavioral Health Concepts 
Inc.’s working documents and copies of this request will be available at our web site, at 
http://www.bhcinfo.com/eqro. We look forward to working with you again this year in 
implementing the External Quality Review. 


Sincerely, 


Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Director, Performance Management Solutions Group 


1. 2004 EQRO Communication Plan 

2. Summary of Performance Improvement Projects to be Validated 

3. Performance Improvement Project Validation General Instructions 

4. Performance Improvement Project Validation Submission Requirements 
5. Summary of Performance Measures to be Calculated 

6. Method for Calculating HEDIS 2004 Performance Measures 

7. Performance Measure Validation General Instructions 

8. Performance Measure Validation Submission Requirements 

9. Encounter Data Validation General Instructions 

10. MCO Compliance General Instructions 


CC: 


Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of Medical Services 


Performance Improvement Project Validation 
General Instructions 


Mail Binder To: 

Attn: External Quality Review Submission 
Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4a 

Columbia, MO 65203 


Due Date: November 23, 2004 


Please refer to Performance Improvement Project Validation Submission Requirements 
and Summary of Performance Improvement Projects to be Validated. 


Performance Measure Validation 
General Instructions 


Mail Binder To: 

Attn: External Quality Review Submission 
Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4a 

Columbia, MO 65203 


Due Date: November 23, 2004 


When applicable, submit one for each of the three measures: 
e Adolescent Immunization Status 
e Adolescent Well Visits 
e Use of Appropriate Medications for People with Asthma 


Note: We are continuing to work with the Division of Medical Services to identify readily 
available documents and data that have been submitted to other entities in the past. For the 
performance measures, the DMS is working to obtain the File Content referenced in the 
legislation (19 CSR 10-5.010) for the HEDIS 2004 measures for the MC+ product and regions 
(HEDIS Data Submission Tool) under the Cooperative Agreement between The Missouri 
Department of Social Services, Division of Medical Services and The Missouri Department of 
Health and Senior Services for Quality Assessment and Improvement. However, if this is not 
available from the DHSS, BHC will request this from MCOs by November 23, 2004. 


Unless otherwise indicated, please send all documents in hard copy, using the enclosed 
binder and tabs. If an item is not applicable or not available, please indicate this in the tab. 


General data submission instructions 

Data file formats all need to be ASCII, and readable in Microsoft Windows environment. 
Use an appropriate delimiter for data that may contain commas or quotation marks. 
Insure that date fields either contain a null value or a valid date. Make all submissions 
using compact disk (CD) formats. Insure that files on the CD are accessible on a 
Microsoft Windows workstation prior to submitting. 


Specific data submission instructions 
You may submit measure-specific files or a single numerator and denominator file. Please 


provide documentation of the measure and file type that is being submitted. 


There should be at least 2 files 
The third file is only for those MCO’s using hybrid methodology. 


HEDIS numerator file for each of the 3 measures. Insure that each file type sent has 
additional documentation identifying record layouts and field definitions. 

HEDIS denominator file for each of the 3 measures. Insure that each file type sent has 
additional documentation identifying record layouts and field definitions. 

For measures calculated using hybrid methodology, a list of all records selected for 
review. 


Please see Performance Measure Validation Submission Requirements and Summary of 
Calculation Methods for Performance Measures. 


Encounter Data Validation 
General Instructions 


Mail To: 


Behavioral Health Concepts, Inc. 
Attn: William Chase 

2716 Forum Blvd., Suite 4a 
Columbia, MO 65203 


Label the package CONFIDENTIAL 


Due Date: This will be requested within ten business days of MCO receipt of 
the encounter sample. 


General data submission instructions 

Data file formats all need to be ASCII, and readable in Microsoft Windows environment. 
Use an appropriate delimiter for data that may contain commas or quotation marks. 
Insure that date fields either contain a null value or a valid date. Make all submissions 


using compact disk (CD) formats. Insure that files on the CD are accessible on a 
Microsoft Windows workstation prior to submitting. 


Specific data submission instructions 
Please provide some documentation on what each electronic file being submitted. 


There should be at least 3 files: 


. File |: Provider mailing address and contact information for sampled claims (service dates 
January |, 2004 to March 31, 2004). This will be used for medical record requests and 
validation of the State medical encounter claims database against the medical record. 
This listing needs to include provider names, clinic manager names, complete address 
information as well as a clear indication of the sampled members receiving services. 
When the sample is drawn, a file will be sent to the MCO with full 
instructions and detailed documentation on what to return to BHC. 


File 2: All other encounters for the selected sample from January |, 2004 to March 31, 
2004 for selected cases, with provider information, from all databases (e.g., inpatient, 
outpatient, medical, dental, home health, pharmacy). This should include only paid claims 
and should be the same layout as that submitted to Verizon. 


This submission should be the same record layout used by the MCO for submitting data 
to Verizon during this same time period. Use the record layout in effect prior to 
October 2004. The "Encounter Claims PAID" submission is designed to replicate the 
data submitted to Verizon from the original submission period. Indicate on the 
submission documentation what you are sending and the file name. 


File 3: All other encounters for the selected sample from January |, 2004 to March 31, 
2004 for selected cases, with provider information, from all databases (e.g., inpatient, 
outpatient, medical, dental, home health, pharmacy). This should include only unpaid 
claims and should be the same layout as that submitted to Verizon. 


This submission should be the same record layout used by the MCO for submitting data to 
Verizon during this same time period. Use the record layout in effect prior to October 


2004. The "Encounter Claims UNPAID" submission is designed to replicate the data 
submitted to Verizon from the original submission period. 


MCO Compliance General Instructions 


These will be determined following BHC review of State compliance review findings. 
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Projects 
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—$ ve 
Definition 


OO 
Performance A project designed to assess and 


Improvement Projects improve processes and outcomes 
of care.... that is designed, 


conducted and reported ina 
shea iobets methodologically sound manner. 


Vice President 


Behavioral Health Concepts, Inc. 
Presented at the Quarterly Meeting of the Quality Assessment and Improvement 
Gr 


St. Mary’s Building, Jefferson City, MO 
Thursday, October 28, 2004 








PIP, QI Activity, and 
Focused Study 


Performance Improvement Project 


Designed to improve processes and outcomes of care . 
over time. Pilot projects 


A comprehensive study with every detail written and Satisfaction survey report 


What is NOT a PIP 


prospectively defined 
Tests the effectiveness of an intervention 
Involves the improvement of quality or access 
Quasi-experimental research design A timeline 
QI Activity : A manual, policy, or procedure 
The intervention being tested by the PIP A pre-authorization manual 
Focused Study 


Is designed to assess processes and outcomes on a 
one-time basis 


A chart or figure 
A workplan 











PIP Elements 


——.———_——_—_ 
Study Topic 

Study Question 

Study Indicators 

Study Population 

Sampling Methods 

Data Collection Procedures 

Improvement Strategies (Intervention/Activity) 
Data Analysis 

Data Interpretation 

Validity 

Sustained Improvement 





Study Topic 


——-- a _$§$§|{|_$§{$§$$§ ars 


What is the problem? 


What is the magnitude of the problem 
(baseline)? 


Is there an intervention to address the 
problem? 


Who does it currently impact? 





PDSA Cycle 


> 
Plan 


Problem identification and quantification 
Barrier analysis 
Potential interventions 
Design study 

Do 
Implement and measure the intervention 
Collect data 

Study 
Conduct analyses 
Interpret data 
Determine if intervention was effective 
Refine intervention 

Act 
Implement refined intervention 
Continue or expand effective intervention 





Selecting Topics 


a _$§$§$§$§$§ rs 


High volume services or diagnoses 
High risk conditions, services, and procedures 


Focus on health promotion, disease 
prevention, acute care, and chronic care 


Based on aclinical or nonclinical intervention 
NOT solely utilization or cost-based 








Sources of Problem 
Identification 


From data collection and analysis of 
client needs 

Analysis of over or under utilization of 
services 

Based on enrollee input (satisfaction 
surveys, grievances, focus groups) 





Sample Study Topic 


> 
Interventions that have been implemented to 
date include system wide collaboration with 

lead coalitions, coordination with school-based 
health clinics, lead case management, and 
provider education. This MCO Is interested in 
addressing the need for parent education on 
blood lead level testing and case management 
interventions in identifying and treating children 
with lead poisoning. 





Considerations 
SE 


Why was this problem chosen? 
What is the potential impact of the 
intervention‘ 

Quality 

Access 

Efficiency 
What are the current barriers to the 
current performance? 


What are the promising approaches to 
change identified in the marketplace 
and literature? 





Study Question 


Should be detailed, specifying the intervention, 
the population, and the expected outcome 


Should be clearly understood by anyone 
Examples: 


Will providing childcare to MC+ Members increase 
the access of parents of young children to prenatal 
care? 

Will adding a mobile case manager for MC+ 
Members increase the access of high risk infants to 
pediatric services? 








Hypotheses 
wT 
Children whose parents received reminder 
cards and lead education materials will be 
more likely to: 
Schedule an EPSDT visit, 
Receive a BLL, 
Be identified with lead toxicity; and 
Less likely to: 
Have sick child visits 
Be hospitalized for chelation therapy 





Logic Model 


> _—_§_ @ @ — arr 
Identifies the conceptualization of the project 


Suggests study design and measures 

What is the logic model; i.e., what is the logic 
behind the performance improvement 
project? 

How and what is expected to be changed? 
What will be done to make change occur? 
Who will be involved in the change? 

How will change be measured? 


How will you know that if change occurred, 
that it is due to the intervention? 











Sample Study 
Questions 


Does parent education and early identification of 
children with blood lead levels greater than 
15y/dL result in better outcomes for children? 


Does sending birthday cards to children before 
their 1st or 2nd birthday with parent education 
materials on the importance of blood lead testing 
result in increased identification of children for 
lead case management? 


Does lead case management result in lower 
blood lead levels and better functional status? 


Are the interventions cost effective? 


Study Design 
Considerations 


What data are available or able to be collected on 
the individuals at pre- and post-intervention? 


What other data need to be collected? 


Is the intervention strong enough to impact the pre- 
and post-intervention measure(s)? 


What are the questions to be answered? 


How will you know whether the intervention worked 
or not? A priori definition. 





Study Indicators 


Implementation Measures 


To assess how well the intervention is being implemented (e.g., number of home 
visits; number of outreach phone calls made) 


To identify the need for modification of the intervention or the delivery of the 
intervention 
Short-Term Measures 
To assess early indices that the intervention is taking place (e.g., service 
utilization, participation, inquiries) 
Intermediate- Term Measures 
To assess shorter term impact of the intervention 
Long-Term Measures (multi-year) to assess impact and outcomes 
Benefit/Cost (in addition to outcome measures) 
Change in performance measure (e.g., HEDIS, MCH Indicators) 
Health Status — physical or emotional 
Functional status 
Quality of life 
Attitude and perception 
Family functioning 
Satisfaction with care, provider, MCO 





Study Design 


——__————_ 


Quasi-experimental research 
Control or comparison groups 
Time series design 
Across providers 
Members who decline services 
Measurement points 
Data to be collected and methods 
Who, what, where, when, and how 
Data analysis plan 





Sample Study 


Indicators 


TT 
Intervention 


Rate of returned mail 


Rate of lead poisoning case management units per member of the 
study population 

Short-Term 
Rate of scheduling of appointments for well-child care in the study 
population 

Intermediate Term 
ead blood lead testing conducted in the study population (goal 


° 
Blood lead level 
Long-Term 
Rate of children identified for lead case management 
Rate of children hospitalized for chelation therapy 
Rate of sick child visits 


Costs of chelation therapy, case management, and birthday 
cards/notification 





Sample Study Design 


oo 
Who is being studied 


Description of intervention 


Description of measures, how, and when they 
will be measured 








Study Population 


| 
The study population to be included in the 
performance improvement project will be 
children enrolled in the MC+ Managed Care 
Program, and their parents. Children will be 
identified two months prior to their 1st or 2nd 
birthday. 





Sampling 


—__ 


Probability 
Every element has a known chance of being 
selected 
Can calculate means, totals, proportions 
Lower samples required 
Small (less than 20) 
Moderate (20 — 50) 
Large to very large (100 or more) 
Nonprobability 
Selection of units based on the judgment of 
how well the sample represents the population 
Proportional 
High standard errors 











Sampling 


Ss Ea 


Probability 
Simple random 
Stratified random 


Based on the subgroups identified for 
comparison 


Cluster 
Nonprobability 
Systematic/Purposive 
Convenience/Judgment 


Sampling 


_ 


Sampling frame 
Sampling units 
Selection process 





Intervention 
Ss Ea 


One or many levels 

System 

Provider 

Child/Family 

MCO Operation (nonclinical) 
Strength (Dosage) 





Data Collection 


————————_———_—__§_@§| rrr 
Baseline data for the previous year on all the 
indicators for the study population will be 
collected and examined on a monthly basis. 
Upon implementation of the intervention, data 
for each indicator for the study population will 
be measured on a monihly basis. Data will be 
entered into MS Excel or the Statistical 
Package for the Social Sciences to create 
quality control charts and chi-square analysis. 











Pilot the Intervention 
(not a PIP) 


Helps estimate the magnitude of the 
change 

Opportunity for failure without a big 
investment 

Become aware of other issues and 
“side effects” 

Learn how to adapt the intervention to 
multiple settings 


System 
Interventions 


School-based testing and exams 

Underwrite urgent care centers, dental offices, and 
clinics 

Ensure specialty physician access for rural areas 
Lay community workers/neighborhood centers 
Transportation vouchers 

Free smoking cessation programs with 
transportation 

Provide MVI with folic acid for all scripts for 
childbearing women 


Outbound calling on weekends and evenings 


Distribute phone cards to members in case 
management 











Provider 


Interventions 
Draw blood lead level at the office, arrange 
for lab pick up 


Have admitting nurse do Lead Risk 
Assessment 


Send nurse practitioner to do EPSDT in the 
home 


Allow billing for education visits 

Nurse home visiting 

Patient educator based at provider office 
Incentives for prenatal visits 
Home-based mental health therapy 


Data Analysis 


+c rr 


A mixed model design 


Statistical quality control charts will be used to measure the pre- and 
ost- intervention effectiveness of the parent education materials and 
ead case management services. Measures of central tendency and 
standard deviation will be used for analysis, with a 1.5 standard 
deviation considered significant. Analyses will be conducted 
separately for 12- and 24- month olds. 
Those who are unable to be reached via mail but remain enrolled in the 
MCO will serve as a control group. The same measures will be 
examined for the intervention group (those who were sent information 
without the mail returned), and the comparison group (those whose 
mail was returned). Chi-square analyses comparing the intervention 
and comparison groups will be conducted on all measures, using the 
95% level of significance. Analyses will be conducted separately for 
12- and 24- month olds. 


A benefit-cost analysis will be conducted to compare the costs of lead 
poisoning case management services and birthday card reminders 
relative to chelation treatment and sick child visits. 


Interventions or 
Not? 


> 


Care coordination 

Case management 
Reminder cards and letters 
Provider education 
Newsletter articles 





Descriptive and 
Inferential Statistics 


> 
Descriptive 


Totals 
Central tendency 
Proportion 
Inferential 
Chi-square (ratios) 
T-tests (means) 
Confidence intervals (variance) 








Confidence Interval Sample Analyses 
i i 
Range of accuracy/sensitivity/certainty Rate of EPSDT Visits, 12 month olds 

95% (1.96 standard deviation units) a 
99% (2.58 standard deviation units) 
99.7% (3.00 standard deviation units) 



























































Sample Analyses Sample Analyses 
> 


Rate of Blood Lead Levels, 12 months 


Trends in all Indicators 























—e— EPSDT Visit s( 12) 
—s=— EPSDTVisits(24) 
BLL (12) 

















—>— BLL (24) 
—*— EBL 
—e— Sick Visits 
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Did it Work? 


TTT 
Conduct analyses as planned 
Examine extraneous factors that may have 
contributed to findings 
Control statistically on post-hoc analyses 
If no effects 

Was the intervention implemented as planned? 


Did the intervention include a large enough number of 
members (or providers, or participants) ? 


Was the intervention STRONG enough? 

Examine the implementation and short-term measures 
Explain 

Modify 





SY 


REMEMBER......... 


it is almost impossible to design the perfect 
study or PIP. Concentrate instead on 
developing a PIP that will make the 
greatest difference for the greatest number 
of your members. 





Discussion of Findings 


rrr 
Discuss each study question 

Discuss the limitations of the study 

Discuss whether the findings are likely to be 
accurate, or if they are related to aspects of 
the intervention or analysis 

Propose methods to refine the project, 
measures, and/or intervention 





Resources 
> 


CMS Protocols 
www.cms.hhs.org.gov/medicaid/managedcare 





Health Care Professionals Network 
http ://wilm-web.com/hcnet/PIFiles/pimain.htm 


Physician Performance Improvement Measurement Sets 
http://www.ama-assn.org/ama/ pub/ category/ 4837.html 





Program Outcomes for Children http://ag.arizona.edu/fcs/cyfernet/nowg/health.html 





Improving Birth Outcomes Toolkit 
http://www.chcs.org/usr_doc/bcaptoolkit.pdf 





AHRQ Nat'l Quality Measures Clearinghouse 
http://www.qualitymeasures.ahrq.gov/resources/measureindex.aspx 





Professional Organizations & Federal Agencies 
http://www. abimfoundation.org/agencymiddle.html 











EXTERNAL QUALITY REVIEW OF MISSOURI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


REFERENCES FOR HEALTH SERVICES RESEARCH METHODS 
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Katz, D. L. (2001). Clinical Epidemiology and Evidenced-Based Medicine: 
Fundamental Principles of Clinical Reasoning and Research. Thousand Oaks, CA: Sage. 


Langley, R. (1970). Practical Statistics. New York: Dover Publications. 


Austin, C. J., and Boxerman, S. B. (1995). Quantitative Analysis for Health Services 


Administration. Ann Arbor, Ml: Health Administration Press. 
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Logic Model Development and Performance Improvement Project 
Planning 


Practical Evaluation of Public Health Programs Workbook 


http://www.phppo.cdc.gov/phtn/Pract-Eval/workbook.asp 


Program Development and Evaluation, University of Wisconsin Extension 
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html#more 


W.J. Kellogg Foundation Logic Model Development Guide 
http://www.wkkf.org/Pubs/T ools/Evaluation/Pub3669.pdf 


Children, Youth and Families Education and Research Network 
http://www.cyfernet.org/ 
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Performance Improvement Project Validation Worksheet 


[Use this or similar worksheet as a guide when validating MCO/PIHP | 
|\Performance Improvement Projects. Answer all questions for each | 
lactivity. Refer to protocol for detailed information on each area. | 


ID of evaluator JR Date of evaluation 





X=Vaakele|ar-] lal rom Maicelmnit-lacela 


MCO/PIHP Name or ID Project Leader Name Telephone Number 


Name of the Performance Improvement Project 


Dates of Study Date Study Initiated 


Type of Delivery System (check all that apply) 










































































Staff Model Network Director IPA 
IPA Organization MCO PIHP 
Number of Medicaid Enrollees Number Medicare Enrollees in 
in MCO or PIHP* MCO or PIHP 
Number of Medicaid Enrollees Total Number of MCO or PIHP 
in the Study —__—— Enrollees in Study 
Number of Members in Study Population of Members in 


Sample Frame 


Number of MCO/PIHP Number of MCO/PIHP 
primary care physicians specialty physicians 
Population of physicians in Number of physicians in study 


sample frame 





Note: DK = Don't Know; NA = Not Applicable 


* Source: Missouri Medicaid Management Information System COLD Reports, State Session MPRI 
Screen, Revised June 25, 2004. Enrollment totals include enrollees with a future start date; 1115, 1915b, 
and Title XXI enrollees as of June 25, 2004. 
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Activity 1: ASSESS THE STUDY METHODOLOGY 





Step 1. Review the selected study topics(s) 











1.1 The topic was selected through data collection Met Partially met Nopwiek 
and analysis of comprehensive aspects of enrollee 
needs, care and services. 









































Not applicable Unable to determine 








Topic or problem 
statement: 


Clinical 
Prevention of an acute or chronic condition High volume services 
































Care for an acute or chronic condition High risk coniditions 








Nonclinical 
Process of accessing or delivering care 














Comments 











1.2 MCO's/PIHP's PIPs, over time, addressed a broad Met Partially met Not met 
spectrum of key aspects of enrollee care and services. 









































Not applicable Unable to determine 








Project must be clearly focused on identifying and correcting deficiences in care or 
services rather than on utilization or cost alone. 
























































Comments 
1.3 MCO's/PIHP's PIPs over time, included all Met Partially met Notiiet 
enrolled populations: 1.e., did not exclude certain 
enrollees such as those with special health care Not applicable Unable to determine 
needs. 
; a Payor 
Demographic description Age 7 si 
—$——<$§=— ace M 
of MC+ population Genter C+ 
Commercial 
Comments 
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Step 2: Review the study question(s) 





2.1 Study question(s) stated clearly in writing. 





Met 


























Study question(s) as 
stated in narrative: 


Comments 





Not applicable 


Partially met 

















Not met 











Unable to determine 


Step 3. Review selected study indicators(s) 





3.1 The study used objective, clearly defined, 
measurable indicators. 

















Indicators (list): 


Comments 





3.2 The indicators measured changes in health status, 
functional status or enrollee satisfaction; or process of 

















care with strong association with improved outcomes. 





Long term outcomes implied or stated: 











Health status: 


Functional status: 


Comments 
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Met Partially met 


Not applicable 


























Met Partially met 


Not applicable 


Yes 























No 





Satisfaction (members): 


Satisfaction (providers): 











Not met 





Unable to determine 











Not met 





Unable to determine 
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Step 4: Review the identified study population 





4.1 MCO/PIHP clearly defined all Medicaid enrollees 
to whom the study questions and indicators are 
























































relevant. 

Demographic description of Age 
MC+ population sampled Conder 
Did it include: 

1115 Yes No 
1915b Yes No 
Children in state custody Yes No 
Consent Decree (Western) Yes No 




















Comments 




































































4.2 If the MCO/PIHP studied the entire population, did 


its data collection approach capture all enrollees to 
whom the study question applied? 


























Methods of identifying {| ytilization data 
participants: = 

| self-identification 
Comments 




















































































































Met Partially met Not met 
Not applicable Unable to determine 
Race —— MCt+ 
Commercial 
Unable to determine NA 
Unable to determine NA 
Unable to determine NA 
Unable to determine NA 
Met Partially met Not met 
Not applicable Unable to determine 
referral 
other 


Step 5: Review sampling methods 





5.1 Sampling technique considered and specified the 
true (or estimated) frequency of occurrence of the 
event, the confidence interval to be used, and the 
margin of the error that will be acceptable. 


Previous findings from: 

















| literature review baseline assessment of indices 





Comments 























Met 











Not applicable 











Other 
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Partially met 























Not met 





Unable to determine 
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5.2 The MCO/PIHP employed valid sampling 














































































































































































































. : : Met Partially met Not met 
techniques that protected against bias. 
Not applicable Unable to determine 
The type of sampling used: 
Probability Nonprobability | Random Simple Stratified 
Convenience Judgment | Quota Cluster 
Comments 
5.3 Sample contained sufficient number of enrollees. Met Partially met Notmet 
Not applicable Unable to determine 
N of enrollees in sampling frame N of sample 


N of participants (i.e., return rate) 


Comments 


Step 6: Review data collection procedures 














6.1 Study design clearly specified the data to be 
collected. 








Met Partially met Not met 



































Not applicable Unable to determine 








Comments 
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Not met 


Not met 


6.2 The study design clearly specified the sources Met Partially met 
of data. 
Not applicable Unable to determine 

Source of data: 
__| Member | Claims | Provider __| Other 

Comments 

6.3 The study design specified a systematic method = @ pet Partially met 

of collecting valid and reliable datathat represents _ 

the entire population to which the study's S] Not applicable Tinableie-detemnme 


indicators apply. 


Comments 


6.4 The instruments for data collection provided for 
consistent, accurate data collection over the time 


periods studied. 


Instrument(s) used: 





__| Medical Record Abstraction Tool 














Survey 





Comments 
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Met Partially met 
Not applicable 
Other 








Not met 








Unable to determine 
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6.5 The study design prospectively specified a data Met Partially met Not met 
analysis plan. 



















































































Not applicable Unable to determine 
Comments 
6.6 Qualified staff and personnel were used to collect Met Partially met Not met 
the data. 
Not applicable Unable to determine 











Name Title 
Role(s) of Project Leader 


Comments 





Step 7: Assess improvement strategies 


7.1 Reasonable interventions were undertaken to Met Partially met Not met 
address causes/barriers identified through data 
analysis and QI processes undertaken. Not applicable Unable to determine 
























































Describe Intervention: 


Comments 
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Step 8: Review data analysis and interpretation of study results 





NA if study is not yet complete 


8.1 An analysis of the findings was performed 
according to data analysis plan. 























Met 











Not applicable 


Partially met 














Not met if study is complete and no indication of a data analysis plan (see step 6.5) 


Comments 


8.2 The MCO/PIHP presented numerical PIP results 
and findings accurately and clearly. 











Are tables and figures labeled? 





Comments 


8.3 The analysis identified initial and repeat 
measurements, statistical significance, factors that 
influence comparability of initial and repeat 
measurement, and factors that threaten internal and 
external validity. 


Indicate time periods of 
measurements: 


Indicate statistical analyses 
used: 


Indicate statistical significance level or 3 
confidence level used: 99% 














Comments 



































Met 








Partially met 





Not applicable 

















Met 

















95% 
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Not applicable 























Partially met 






































Not met 





Unable to determine 











Not met 





Unable to determine 


Labeled clearly, accurately? 











Not met 





Unable to determine 


Unable to determine 
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8.4 Analysis of study data included an interpretation Met Partially met Not met 
of the extent to which its PIP was successful and 
follow-up activities. Not applicable Unable to determine 




















Limitations described: 


Conclusions regarding the success 
of the interpretation: 


Recommendations for follow-up 
Comments 








Step 9: Assess whether improvement ts "real" improvement 


Note: NA only if study period is not yet complete: otherwise "Unable to Determine" or "No" 











9.1 The same methodology as the baseline Met 































































































































































































































































































Partially met Not met 
measurement was used when measurement was = 
repeated. Not applicable ‘| Unable to determine 
Same source of data yes No Not applicable Unable to determine 
Same method of data collection yes No Not applicable Unable to determine 
Same participants examined yes No Not applicable Unable to determine 
Same tools used yes No Not applicable Unable to determine 
Comments 
9.2 There was a documented, quantitative Met ) Partially met 'm] Not met 
improvement in process or outcomes of care. re _ 

Not applicable Unable to determine 
increased decreased 
Statistical significance Clinical significance 
Comments 
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9.3 The reported improvements in performance have Met Partially met Not met 
"face" validity: i.e., the improvement in performance 
appears to be the result of the planned quality Not applicable Unable to determine 


improvement intervention. 


Degree to which the intervention was ll No relevance ‘| Small el Fair | High 
the reason for change: 






















































































Comments 
9.4 There is statistical evidence that any observed Met Partially met Noimet 
performance improvement is true improvement. 
Not applicable Unable to determine 
Weak ‘ Moderate Strong 
Comments 


Step 10: Assess sustained improvement 




















Met Partially met Not met 














10.1 Sustained improvement was demonstrated 
through repeated measurements over comparable Not applicable Daanicradetemnine 
time periods. 





























Comments 
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ACTIVITY 3: EVALUATE OVERALL VALIDITY AND RELIABILITY OF STUDY 
RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS AND 








RECOMMENDATIONS 
Conclusions 
Recommendations 
Check one: 
High confidence is reported Low confidence level is reported in MCO/PIHP PIP results 




















!, Moderate confidence is reported MCO/PIHP PIP results ‘| Reported MCO/PIHP PIP results not credible 














Not Applicable, study not complete 
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Behavioral Health Concepts, Inc. 
Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573)446-0405 
(573)446-1816 (fax) 
(866)463-6242 (toll-free) 
www.bhcinfo.com 








Friday, November 12, 2004 

Re: 2004 External Quality Review Performance Measure Submission Request 

Dear Quality Managers: 

In response to some questions that have arisen following the teleconference orientation and our written 
request for files regarding the three performance measures to be validated for the 2004 External 


Quality Review, we are providing the following clarification. As you know, we are validating three 
measures from the HEDIS 2004 measures for the MC+ Managed Care Program population: 


7 Use of Appropriate Medications for People with Asthma 
7 Adolescent Well-Care Visits 
7 Adolescent Immunization Status 


Use of Appropriate Medications for People with Asthma 
Denominators 


Provide all cases selected for denominators 
Include date of birth, enrollment start date, qualifying event/diagnosis (see pages 127 and 128, 
HEDIS 2004 Technical Specifications), and exclusions 
Numerators 
Provide all positive cases or “hits” with National Drug Code (NDC) 
Include date of birth, enrollment start date, qualifying event/diagnosis, and exclusions 


Adolescent Well-Care Visits 

Denominators 
Provide all cases selected for denominators 
Include date of birth and enrollment start date 


Numerators 
Administrative Method 
Provide all positive cases (“hits”), with CPT and ICD-9CM codes 
Hybrid Method 
Provide all positive cases (“hits”), with CPT and ICD-9CM codes for administrative “hits” 
Provide all cases selected for medical record review 
Include an indicator for status of the medical record (reviewed/not reviewed, 
substitution) 
Include an indicator for positive (“hits”) and negative cases 


Adolescent Immunization Status 
Denominators 
Provide all cases selected for denominators 
Include date of birth and enrollment start date 
Numerators 
Administrative Method 
Provide all positive cases (“hits”), with CPT and ICD-9-CM codes for administrative “hits” 
Include an indicator for positive cases (“hits”) using other administrative data (e.g., 
MOHSAIC) 
Include cases with exclusions and the contraindication or ICD-9-CM code 


Hybrid Method 
Provide all positive cases (“hits”), with CPT and ICD-9-CM codes for administrative “hits” 
Provide all cases selected for medical record review 
Include an indicator for status of the medical record (reviewed/not reviewed, 
substitution) 
Include an indicator for positive (“hits”) and negative cases 
Include cases with exclusions and the contraindication or ICD-9-CM code 


Behavioral Health Concepts, Inc. 
Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573)446-0405 
(573)446-1816 (fax) 
(866)463-6242 (toll-free) 
www.bhcinfo.com 








January 3, 2005 
Re: 2004 External Quality Review Performance Measure Validation Protocol (for hybrid measures only) 
Due Date: February 4, 2004 


We have reviewed your MCOs numerator files for the medical record review of the HEDIS 2004 
Adolescent Immunization measure. Enclosed is a listing of cases sampled for review. We are requesting 
copies of medical records for the sampled cases that contributed to the numerator. Please forward 
copies of the medical records to Behavioral Health Concepts, Inc. (BHC) for the Performance Measure 
Validation Protocol with the case listings. Send them to BHC at the address listed above, and mark the 
package as confidential. 


If you have any questions, Ms. Prater, the EQRO Assistant Project Director can be reached at (573) 446- 
0405. 


Thank you, 


Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Director, Performance Management Solutions Group 


Cc: 


Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of Medical 
Services 
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Tuesday, April 26, 2005 
Re: 2004 External Quality Review Performance Measure Validation Protocol 


Due Date: May 16, 2004 
Dear QA & I/UM Coordinator: 


The CMS Final Protocol for the Validation of Performance Measures provides for a process whereby the 
MCO reviews the summary findings of the EQRO regarding the Performance Measure Validation. The 
findings will be incorporated into the External Quality Review Organization report to be submitted 60 
days after final ratings are received by the State Medicaid Agency. In accordance with Option | of the 
Protocol, | have enclosed several documents for your review. Please send any comments or corrections 
in writing to me no later than May 16, 2004 at the above address. The following are the enclosed 
documents: 


e Objectives and Technical Methods 

e Final Performance Measure Validation Worksheets 

e Attachments 

e Validation of Performance Measure summary for your MCO 
Thank you, 


Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Director, Performance Management Solutions Group 


CC: 


Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of Medical 
Services 
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Appendix 5: Validation of Encounter Data 
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Recommended Encounter Data Validation Criteria 
Data Element Expectation BME lin mater 
Enrollee ID Should be valid as 100% valid 
found in the State's 
eligibility file. 


Principal Diagnosis Well-coded lead- > 90% non-missing and valid 
related diagnoses codes. 
(or well-child visit) 

Date of Service Dates should be If looking at a full year of data 
evenly distributed 5-7% of the records should be 
across time distributed across each month. 


Unit of Service (Quantity) The number should 98% non-zero 
be routinely coded. <70% should be one if CTP 


code in range of 99200-99215, 
99241-99291 


Procedure Code This is a critical 99% present (not zero, blank, 
element and should — 8- or 9-filled). 100% should be 
always be coded. valid, State-approved codes. 
Will be assessed There should be a wide range of 
only for presence of | procedures with the same 
code except for frequency as previously 
lead-related codes encountered. 
which will be 
validated with 
medical records. 

Source: Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data:: Second Edition. 





Performance Management Solutions Group 408 
a Division of Behavioral Health Concepts, Inc. 

2/16 Forum Blvd., Suite 4 T: 573.446.0405 

Columbia, MO 65203 F: 573.446.1816 


www. bhcinfo.com/PMSG E-mail: info@bhcinfo.com 


EXTERNAL QUALITY REVIEW OF MissouR!I MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


ENCOUNTER CLAIMS DATA REQUEST 

Please pull Encounter Claims data for the 2004 EQR. Report all Encounter Claims on file for dates 
of service January |, 2004 — March 31, 2004. Include all claim types, Inpatient, Outpatient, Dental, 
Medical, Drug, and Home Health. Use the layouts from last year’s request with one exception. On 
the Outpatient layout, add a separate field for Revenue Code. If there is no revenue code on the 
claim, leave the field blank. 

Please put data on a CD. 

The layouts for last year’s request are below. Please send an updated Outpatient Layout with the 


Revenue Code field added. 


01 OUTPATIENT-RECORD. 
05 OUTPAT-HEADER-COMMON. 
10 OUTPAT-CLAIM-TYPE PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-SUBMIT-TYPE PIC X(01). 
10 FILLER PIC X VALUE '@". 
10 OUTPAT-CLAIM-STATUS PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-TOT-PAID-AMT PIC $$$$$$$9.99. 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-PROCESSED-RECIP-ID PIC X(08). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-RECIP-LAST-NAME PIC X(14). 
10 FILLER PIC X VALUE '@". 
10 OUTPAT-RECIP-FIRST-NAME _ PIC X(12). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-RECIP-BIRTHDATE PIC 9(08). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-RECIP-SEX-CODE PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-RECIP-CNTY-CD PIC. 9(3). 
10 FILLER PIC X VALUE '@". 
10 OUTPAT-RECIP-RACE-CODE PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-PATIENT-ACCT-NUM PIC X(20) VALUE SPACES. 
10 FILLER PIC X VALUE '@". 
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10 OUTPAT-HEADER-EOB 


OCCURS 2 TIMES. 
15 OUTPAT-EOB PIC X(03) VALUE SPACES. 
15 FILLER PIC X(01) VALUE '@'. 
10 OUTPAT-PROV-NUM PIC X(09). 
10 FILLER PIC X VALUE '@' 
10 OUTPAT-HEADER-DIAGNOSIS 
OCCURS 5 TIMES. 
15 OUTPAT-DX PIC X(05) VALUE SPACES. 
15 FILLER PIC X(01) VALUE '@'. 
++*4***|NSURANCE COVERAGE CODE (TPL)***ttetetesnenenes 
10 OUTPAT-KEY-TPL-IND PIC X(01). 
10 FILLER PIC X VALUE '@' 


05 OUTPAT-DETAIL. 
10 OUTPAT-DTL-LINE-NUM PIC 9(02). 


10 FILLER PIC X VALUE '@'. 
10 OUTPAT-FIRST-DT-SVC PIC 9(08). 
10 FILLER PIC X VALUE '@". 

10 OUTPAT-LAST-DT-SVC PIC 9(08). 
10 FILLER PIC X VALUE '@'. 

10 OUTPAT-PLACE-OF-SVC PIC X(02). 
10 FILLER PIC X VALUE '@". 

10 OUTPAT-TYPE-SVC PIC X(01). 

10 FILLER PIC X VALUE '@'. 

10 OUTPAT-UNITS-SVC PIC 9(05). 

10 FILLER PIC X VALUE '@". 

10 OUTPAT-RECIP-ME-CODE PIC X(02). 
10 FILLER PIC X VALUE '@'. 


10 OUTPAT-DTL-PRCD. 
15 OUTPAT-DTL-PROC PIC X(05). 
15 OUTPAT-DTL-PROC-MOD-P_ PIC X(02). 
15 OUTPAT-DTL-PROC-MOD-I PIC X(02). 
10 FILLER PIC X VALUE '@' 
10 OUTPAT-PERF-PROV-MCARE-NUM PIC X(16). 
10 FILLER PIC X VALUE '@'. 
10 OUTPAT-DTL-DIAG-CODE PIC X(05). 
10 FILLER PIC X VALUE '@' 
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10 OUTPAT-DTL-PAID-AMT PIC $$$$$$9.99. 


10 


FILLER PIC X VALUE '@'. 


**EREEECT AIMS REJECTION MESSAGE **44t ttt teeee ee eRe REER ES 
10 OUTPAT-DTL-EOB-CODE PIC X(03). 


10 
10 


FILLER PIC X VALUE '@’. 
FILLER PIC X(24). 


01 INPATIENT-RECORD. 
05 INPAT-HEADER-COMMON. 


10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


INPAT-CLAIM-TYPE PIC X(01). 
FILLER PIC X VALUE '@'. 
INPAT-SUBMIT-TYPE PIC X(01). 
FILLER PIC X VALUE '@'. 
INPAT-CLAIM-STATUS PIC X(01). 
FILLER PIC X VALUE '@'. 
INPAT-TOT-PAID-AMT PIC $$$$$$$9.99. 
FILLER PIC X VALUE '@' 
INPAT-PROCESSED-RECIP-ID PIC X(08). 
FILLER PIC X VALUE '@' 
INPAT-RECIP-LAST-NAME PIC X(14). 
FILLER PIC X VALUE '@'. 
INPAT-RECIP-FIRST-NAME PIC X(12). 
FILLER PIC X VALUE '@'. 
INPAT-RECIP-BIRTHDATE PIC 9(08). 
FILLER PIC X VALUE '@'. 
INPAT-RECIP-SEX-CODE PIC X(01). 
FILLER PIC X VALUE '@". 
INPAT-ME-CODE PIC X(02). 
FILLER PIC X VALUE '@' 
INPAT-RECIP-CNTY-CD PIC 9(3). 
FILLER PIC X VALUE '@". 
INPAT-RECIP-RACE-CODE PIC X(01). 
FILLER PIC X VALUE '@". 


INPAT-PATIENT-ACCT-NUM PIC X(20) VALUE SPACES. 
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10 FILLER PIC X VALUE '@'. 
10 INPAT-ADMIT-TYPE PIC X(01). 
10 FILLER PIC X VALUE '@' 
10 INPAT-ADMIT-DT PIC 9(08). 
10 FILLER PIC X VALUE '@'. 
10 INPAT-MED-DSCHG-DT PIC 9(08). 
10 FILLER PIC X VALUE '@' 
10 INPAT-BILL-TYPE PIC X(03). 
10 FILLER PIC X VALUE '@' 
10 INPAT-PATIENT-STAT PIC X(02). 
10 FILLER PIC X VALUE '@' 
10 INPAT-HEADER-EOB 
OCCURS 2 TIMES. 
15 INPAT-EOB PIC X(03) VALUE SPACES. 
15 FILLER PIC X(01) VALUE '@'. 
10 INPAT-PROV-NUM PIC X(09). 
10 FILLER PIC X VALUE '@' 
10 INPAT-HEADER-DIAGNOSIS 
OCCURS 5 TIMES. 
15 INPAT-DX PIC X(05) VALUE SPACES. 
15 FILLER PIC X(01) VALUE '@'. 
++*4***|NSURANCE COVERAGE CODE (TPL)***tetetetnenenenes 
10 INPAT-KEY-TPL-IND PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 INPAT-FIRST-DT-SVC PIC 9(08). 
10 FILLER PIC X VALUE '@' 
10 INPAT-LAST-DT-SVC PIC 9(08). 
10 FILLER PIC X VALUE '@". 
05 INPAT-DETAIL. 
10 INPAT-DTL-LINE-NUM PIC 9(02). 
10 FILLER PIC X VALUE '@". 
10 INPAT-REVENUE-CD PIC X(03). 
10 FILLER PIC X VALUE '@'. 
10 INPAT-UNITS-SVC PIC 9(05). 
10 FILLER PIC X VALUE '@' 
10 INPAT-DTL-PAID-AMT PIC $$$$$$9.99. 
10 FILLER PIC X VALUE '@'. 
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**EREEECT AIMS REJECTION MESSAGE ttt ttre tee aan teen eee es 


10 INPAT-DTL-EOB-CODE PIC X(03). 
10 FILLER PIC X VALUE '@". 
10 FILLER PIC X(31). 


01 PHARMACY-RECORD. 
05 PHARMACY-HEADER-COMMON. 
10 PHARM-BIN-MO-MEDICAID PIC 9(06) VALUE 004047. 
10 FILLER PIC X VALUE '@' 
10 PHARM-VERSION-NUM PIC X(02) VALUE '3C'. 
10 FILLER PIC X VALUE '@'. 
10 PHARM-SUBMIT-TYPE PIC X VALUE SPACES. 
10 FILLER PIC X VALUE '@' 
10 PHARM-TRANSACTION-CODE _ PIC X(03) VALUE SPACES. 
10 FILLER PIC X VALUE '@'. 
10 PHARM-PROV-NUM PIC X(09). 
10 FILLER PIC X VALUE '@'. 
10 PHARM-PROV-NABP-NUM PIC X(07) VALUE SPACES. 
10 FILLER PIC X VALUE '@' 
10 PHARM-PROCESSED-RECIP-ID PIC X(08). 
10 FILLER PIC X VALUE '@' 
10 PHARM-KEY-TPL-IND PIC X(01). 
10 FILLER PIC X VALUE '@'. 
10 PHARM-FIRST-DT-SVC PIC 9(08). 
10 FILLER PIC X VALUE '@' 
10 PHARM-RECIP-LAST-NAME PIC. X(14). 
10 FILLER PIC X VALUE '@". 
10 PHARM-RECIP-FIRST-NAME PIC X(12). 
10 FILLER PIC X VALUE '@'. 
05 PHARMACY-DATA-RECORD. 
10 PHARM-PRESCRIPTION-NUM PIC X(07). 


10 FILLER PIC X VALUE '@'. 
10 PHARM-REFILL-IND PIC X(01). 
10 FILLER PIC X VALUE '@’. 
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10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


PHARM-DRUG-QTY-DSPNS PIC 9(05). 


FILLER PIC X VALUE '@' 
PHARM-DAYS-SUPPLY PIC 9(03). 
FILLER PIC X VALUE '@'. 
PHARM-COMPOUND-IND PIC X(01). 
FILLER PIC X VALUE '@' 
PHARM-DRUG-NDC-CODE PIC X(11). 
FILLER PIC X VALUE '@' 
PHARM-PRESCRIBE-PROV-NUM _ PIC X(09). 
FILLER PIC X VALUE '@' 
PHARM-DRUG-KEYED-EPSDT-IND PIC X(01). 
FILLER PIC X VALUE '@'. 
PHARM-TOT-CLM-PMT PIC $$$$$$9.99. 
FILLER PIC X VALUE '@' 

FILLER PIC X(11) VALUE SPACES. 
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DATA REQUEST FOR RESEARCH AND EVALUATION 
Please run the following request for the 2004 EQR. 


Provide an unduplicated file of recipients who were locked in and eligible at any time between 


January |, 2004 — March 31, 2004. Do not include recipients whose lockin start date = lockin stop 


date. Please provide data on a CD. 


Valid health plan lockin numbers are: 


817919905 
817920002 
817919608 
818101305 
818920407 
818287708 
818280109 
818287807 
815337407 


Please report the following fields: 


DCN 

Recipient Name 
Recipient Address 
Recipient Street 
Recipient City 
Recipient State 
Recipient Zip 
Recipient Phone Number 
Casehead Name 

DOB 

Sex/Gender Indicator 
County Codes 

Race 

Ethnicity 

Category of Assistance 
Language Indicator 
SSN 


AD HOC REQUEST FOR DIAGNOSIS CODES 


Provide a listing and description of diagnosis codes currently active in the Diagnosis file. Please put 


data ona CD. 
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December 6, 2004 


Re: 2004 External Quality Review Encounter Data Validation Protocol 
Due Date: December 31, 2004 


Enclosed is a list of the materials we received from your organization in response to our first request for 
the 2004 External Quality Review, as of Friday, December 03, 2004. We will review the information we 
have available to conduct the validation of performance measures and the validation of performance 
improvement projects, and will follow-up if we need clarification on specific items. 


Enclosed is a CD-ROM of a file containing the sample of encounters with patient identifying information. 
BHC will use this sample to request medical records from providers. We will send the request to 
providers no later than January 7, 2004, so the prompt return of the data requested below will allow 20 
business days for providers to gather and submit records. BHC does not provide reimbursement for the 
cost of photocopying or mailing records. Records will be required to be submitted no later than 
February 4, 2005. Records not received by that date will be considered undocumented encounters. 
BHC will not be responsible for incorrect provider addresses or provider information. 


In the past, several MCOs have expressed a preference for sending an advance communication to 
providers and following-up with them during the medical record submission process. Feel free to send 
the enclosed list of providers a notification of the upcoming requirement to help them prepare to submit 
records. It is assumed that medical records are complete and that encounters were documented. 
Therefore, providers are not to be instructed or otherwise encouraged to change their medical record 
documentation for the purpose of this review. 


As in the past two years, BHC will provide a status report indicating on the submission rate of records 
during the review process to facilitate MCO follow-up with providers. At the time we send out the 
medical record request, we will also send a copy of the materials to each MCO. These will include a 
letter from Sandra Levels, instructions on the submission, and information regarding federal and state 
requirements for adherence to HIPAA and the External Quality Review. 


As discussed with quality managers individually, with MCO staff in the technical assistance 
meeting over teleconference, and at the All-Plan meeting, we are requesting the following 
information from each MCO for the enclosed encounters: 


|. Complete and current contact information for providers, including clinic manager name 
when it is available. 

2. All additional paid encounters for the individuals associated with each sampled claim. 

3. All unpaid encounters the individuals associated with each sampled claim. 


Identifiers for paid and unpaid encounters and claim type are to be included on each record. 
The record layouts for paid and unpaid claims are to be consistent with the record layout in 
effect prior to October, 2004. 


Once we have selected samples from the hybrid HEDIS measures calculated by your MCO, we 
will forward them to you to obtain and forward the medical record to BHC no later than 
February 4, 2004 for the performance measure validation process. If you have questions about 
the file submission process for the encounter sample, contact William Chase, Director of 
Research and Information Systems. For questions about the medical record review process, 
contact Mona Prater, our new Assistant Project Director. She will be coordinating and leading 
the compliance review process and facilitating project coordination. Both Mr. Chase and Ms. 
Prater can be reached at (573) 446-0405. 


Thank you, 


Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Director, Performance Management Solutions Group 


Encl: 


|. Listing of submissions received for the validation of performance 
improvement projects and performance measures for 2004 External Quality 
Review 

2. Encounter Validation Submission Instructions 

3. CD-ROM with sample of encounters for encounter data validation 


Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of 
Medical Services 


MC+ MCO Administrator 


Encounter Data Validation Submission Instructions 
Mail To: 


Behavioral Health Concepts, Inc. 
Attn: William Chase 

2716 Forum Blvd., Suite 4a 
Columbia, MO 65203 


Label the package CONFIDENTIAL 


Due Date: December 31, 2004 


General data submission instructions 

Data file formats all need to be ASCII, and readable in Microsoft Windows 
environment. Use an appropriate delimiter for data that may contain commas 
or quotation marks. Insure that date fields either contain a null value or a valid 
date. Make all submissions using compact disk (CD) formats. Insure that files 
on the CD are accessible on a Microsoft Windows workstation prior to 
submitting. 


Specific data submission instructions 
Please provide documentation for each electronic file being submitted. The CD 


containing the sample includes a Microsoft Excel document with several sheets. 
One sheet includes some instructions, another sheet has the 100 specific 
sample encounters, and another sheet includes specific fields for provider 
address information your MCO will need to provide. 


There should be at least 3 files: 


File |: Provider mailing address and contact information for sampled claims 
(service dates January |, 2004 to March 31, 2004). This will be used for medical 
record requests and validation of the State medical encounter claims database 
against the medical record. This listing needs to include provider names, clinic 
manager names, complete address information as well as a clear indication of the 
sampled members receiving services. 


The CD contains a Microsoft Excel file with 100 sample encounters. The data included 
for the sample includes this type of information. 


Field Name 


PROCESSED-RECIP-ID 


Field contents 


The Missouri Medicaid recipient identification number. 








RECIP-LAST-NAME 


Entire name may be entered. Only the first two letters of the recipient's 
last name and the first letter of the recipient's first name will be verified 
against the recipient's Medicaid enrollment records. The plan must send 
a minimum of two characters for the last name and one character for the 
first name. 


ee ae Entire name may be entered. Only the first two letters of the recipient's 


last name and the first letter of the recipient's first name will be verified 
against the recipient's Medicaid enrollment records. The plan must send 
a minimum of two characters for the last name and one character for the 
first name. 


RECIP-BIRTHDATE Recipient date of birth 





RECIP-SEX-CODE 
| Male 


2 Female 





FIRST-DT-SVG This is the first date the service was performed. 





2. File 2: All other encounters for the selected sample from January |, 2004 to 
March 31, 2004 for selected cases, with provider information, from all databases 
(e.g., inpatient, outpatient, medical, dental, home health, pharmacy). This should 
include only paid claims and should be the same layout as that submitted to 
InfoCrossing. 


This submission should be the same record layout used by the MCO for 
submitting data to InfoCrossing during this same time period. Use the record 
layout in effect prior to October 2004. The "Encounter Claims PAID" 
submission is designed to replicate the data submitted to InfoCrossing from 
the original submission period. Indicate on the submission documentation 
what you are sending and the file name. 


3. File 3: All other encounters for the selected sample from January |, 2004 to 
March 31, 2004 for selected cases, with provider information, from all databases 
(e.g., inpatient, outpatient, medical, dental, home health, pharmacy). This should 
include only unpaid claims and should be the same layout as that submitted to 
InfoCrossing. 


This submission should be the same record layout used by the MCO for 
submitting data to InfoCrossing during this same time period. Use the record 
layout in effect prior to October 2004. The "Encounter Claims UNPAID" 
submission is designed to replicate the data submitted to InfoCrossing from the 
original submission period. 
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January 7, 2005 


Re: Provider Medical Record Request for the 2004 External Quality Review Encounter Data 
Validation Protocol 


Due Date: February 4, 2004 


Enclosed is a copy of the information sent to providers by Behavioral Health Concepts, Inc. 
(BHC), whose claims were selected for review in the Encounter Data Validation Protocol. 
Medical records received by February 4, 2004 will be included in the review process. 


As with the past three years, BHC will update MCOs during the record review submission 
status of providers and send reminder cards to providers that have not submitted records 
within two weeks. 


Some MCOs in the past have chosen to facilitate the process of medical record submission for 
their providers. If you are submitting records on behalf of providers, please separate the 
records using the pre-printed face sheets given to the provider by BHC, or supplied on the 
enclosed CD-ROM. 


If you have any questions, Ms. Prater, the EQRO Assistant Project Director can be reached at 
(573) 446-0405. 


Thank you, 


Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Director, Performance Management Solutions Group 


Encl: 

CD-ROM with face sheets for medical record requests 
Letter from Sandra Levels 

Instructions to providers 


CC; 
Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of 
Medical Services 


MC+ MCO Administrator 


Please review the encounter claims in which there was more than one provider (as shown in 


C7 ee Pd 


Behavioral Health Concepts, Inc. the shaded rows) and indicate on the spreadsheet (placing an “x” in the designated column) 


INE Dee Ray Renata then rs ear onueere cgrlairneeen ne Oa Soe reba which provider is associated with each randomly selected claim (shown in the white rows). 
(573)446-1816 (fax) 
(866)463-6242 (toll-free) Please return the completed file to BHC by January 28, 2005. 


www.bhcinfo.com 








If you have any questions about this request, please contact me at (573) 446-0405. 


Wednesday, January 19, 2005 Sincerely, 
Re: Provider Medical Record Request for the 2004 External Quality Review 
Encounter Data Validation Protocol Janet S. Reed, PhD, MHA 
Vice President, Behavioral Health Concepts, Inc. 
Thank you for your assistance in obtaining provider contact information for the 100 randomly Director, Performance Management Solutions Group 
selected medical encounters for MC+ Managed Care Program Recipients. We are in the 
process of tracking the receipt of medical records for the Encounter Data Validation Protocol, Encl: 
and will be sending reminder cards this week to providers who have not yet responded. There CD-ROM with MSExcel 2000 file 
are two issues for which we are requesting additional assistance. Patient Face Sheets and Undeliverable Provider Addresses 


CC: Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, 
Division of Medical Services 
MC+ MCO Administrator 


> First, we have enclosed a list of providers whose medical record requests were 
returned undeliverable after several attempts to find correct addresses. We now need 
your assistance in obtaining these medical records. Enclosed is a list of these providers, 
as well as green face sheets with patient and provider information to be included with 
the medical record. Please have these records with the face sheets forwarded to our 
office by February 4, 2005 for the medical record review portion of the Encounter 
Data Validation Protocol. 


> Second, in an number of cases, there was more than one provider associated with a 
particular patient on the specified service date, resulting in more providers than 
encounter claims. We would like to determine which providers are associated with 
each of the randomly selected encounter claims. 


Enclosed is a CD-ROM with an MS Excel 2000 database file containing the randomly selected 
encounter claims for the 2004 External Quality Review Encounter Data Validation medical 
request. The file also contains the provider contact information submitted by your MCO in 
response to our request to append provider contact information for the selected encounters. 
The encounter claims randomly selected by BHC are shown in white, with the providers 
associated with the patient for the date of the encounter claim shown in gray. Not all 
encounters selected had more than one provider. We have displayed only those encounter 
claims for which there was more than one provider listed. 


We have supplied the same list of randomly selected encounters, with additional data fields for 
each randomly selected encounter for the purpose of matching one provider with each selected 
encounter claim. For example, we have added the first diagnosis and procedure code as well as 
other data fields for each encounter claim. 


Behavioral Health Concepts, Inc. 





March 30, 2005 

RE: 2004 Encounter Data Validation 

Due date: April 8, 2005 

Enclosed on a CD-ROM is an MS Excel 2000 spreadsheet containing patient identifying 
information from some of the claims selected for 2004 EQR encounter data validation. We 
need your assistance in identifying a single provider associated with each of the claims listed. 


We are interested in obtaining the contact information for the providers based on the Internal 
Control (ICN) numbers for each claims. The file contains the following data fields: 


Data Provided by BHC, Inc. 




















Field name from State database Field name descriptor 
PROCESSED_RECIP_ID Recipient Medicaid (Departmental Control Number) 
ICN Internal Control Number 

RECIP_FIRST_ NAME Recipient First Name 

RECIP_LAST NAME Recipient Last Name 

FIRST DT SVC First Date of Service 

RECIP_BRITHDATE Recipient DOB 

DX_| First Diagnosis Code 

DTL_PROC Procedure Code 








Please add to this file the following information on providers that is associated with the 
provided Internal Control Number (ICN). 


Provider First Name 
Provider Last Name 
Provider Title 

Provider First Address 
Provider Second Address 
Provider City 

Provider State 

Provider ZIP Code 


Thank you very much for your assistance in facilitating the process of encounter data validation. 





Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573)446-0405 
(573)446-1816 (fax) 

(866)463-6242 (toll-free) 

www.bhcinfo.com 


If you have any questions about this, please contact me at (573) 446-0405 or via e-mail 


mchumak@bhcinfo.com. 


Sincerely, 


Mariya A Chumak 
Research Associate 
Behavioral Health Concepts, Inc. 


Encl: 
CD-ROM with Excel 2000 file 


CC: 
Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social Services, Division of 
Medical Services 


MC+ MCO Administrator 


EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


CRITICAL FIELD VALIDATION PARAMETERS 


There is one table for each claim type. The first column represents the name of the field from the Missouri Medicaid system, 
with a brief descriptor for the field name. The second column shows the length of the contents of the field, and any specified 
formats for the field. The third column is the type of information contained in the data string (e.g., aloha = letters; numeric = 
numbers; alohanumeric = a combination of letters and numbers). The fourth column is the State definition of valid codes, from 
the Data Dictionary provided and Health Plan Record Layout Manual and adapted in the tables below as they apply to each 
claim type. 


Inpatient Claim Validation, UB-92 Layout 
Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 


I NPAT-CLAI M- 1 Alpha This field indicates the claim type that has been 
TYPE Submitted by the Health Plan. Valid values are: 
Inpatient Claim 
Type | = Inpatient 

[Table C-97] 





I NPAT- 
PROCESSED- Numeric The Missouri Medicaid recipient identification number. | OOOOOOO! - 
RECIP-ID 99999999 
Recipient ID The Missouri Medicaid recipient identification number. 

Allowed characters are 0 through 9. 

NOTE: Reference the recipient's Medicaid card for the 

correct Medicaid identification number. 

[Table C-1] 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 
INPAT-ADMIT- 
TYPE Numeric Admission Type [Table C-6] See list of 


Admission. Type values at left. 
The only valid values are: 


The only valid values are: 
1 = Emergency 


Elective 
Newborn 
= Information Not Available 


Required for Inpatient claims. 


INPAT-ADMIT-DT UB-92 CLAIMS - UB-92 dates are in MMDDYY format 
Admission Date 8 Alphanumeric 01/01/2004 - 


UB-92 dates The date the recipient was admitted to the hospital. 03/31/2004 


are in (Translated from aca ae | 7 
MMDDYY Julian calendar) This field is required for Inpatient claims. 


format The date the recipient was admitted to the hospital. 
This date cannot exceed the current date. 


[Table C-16] 


I NPAT-MED- : 
DSCHG-DT 8 Alphanumeric The date the recipient was discharged from the 01/01/2004 - 


UB-92 dates i i is still i i 
Discharge Date sali (rraneiaked from hospital. If the patient Is still in the hospital, the latest | 03/31/2004 
date of service that applies to the claim. 


MMDDYY Julian calendar) 
format [Table C-16] 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 
INPAT-BILL-TYPE 
Bill Type 


Numeric Valid bill type codes are: 110 - 119; or 


11x 120 - 129 


12x 
+8x4Not valid for inpatient) 
[Table C-43] 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data 

[Number of 

Characters] 

and Format 


Valid Values 


I NPAT-PATIENT- 
STAT Numeric The code that represents the condition under which See list of 


the recipient was discharged. values at left. 
Ol Home 

O2 Hospital 

03 Skilled Nursing Facility (SNF) 

04 Intermediate Care Facility (1 CF) 

05 Institution (Inst) 

O06 Home Health Agency (HHA) 

O07 Left 

08 Other 

20 Death 

30 Still A Patient 

50 Discharge from Hospice to Home 

51 Discharge from Hospice to Another Medical Facility 
62 Discharged/transferred to an inpatient 
rehabilitation facility (IRF) including rehabilitation 
distinct part units of a hospital 

64 Discharged/transferred to a nursing facility certified 
under Medicaid but not certified under Medicare 

65 Discharged/transferred to a psychiatric hospital or 
psychiatric distinct part unit of a hospital 


Patient Discharge 
Status 


Required for Inpatient and Outpatient- Hospice claims 
only. 


[Table C-7] 
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Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 
INPAT-DX Alphanumeric, The ICD-9 diagnosis code of the recipient's primary 
3-5 left justified diagnosis. If the dia gnosis code has less than five see DMS 
characters, the data must be left justified and fill the Diagnosis 
remaining positions to the right with spaces. Codes 





Any decimal point needed in the diagnosis code is 
implied and must not be included. 


Required. 
[Table C-17] 
INPAT-DX Alphanumeric, The |CD-9 diagnosis code of the recipient's primary 
a8 left justified diagnosis. If the dia gnosis code has less than five see DMS 


characters, the data must be left justified and fill the Diagnosis 
remaining positions to the right with spaces. Codes 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


Optional 
[Table C-17] 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 
INPAT-DX Alphanumeric, The ICD-9 diagnosis code of the recipient's primary 
3-5 left justified diagnosis. If the dia gnosis code has less than five see DMS 
characters, the data must be left justified and fill the Diagnosis 
remaining positions to the right with spaces. Codes 





Optional 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 
INPAT-DX Alphanumeric, The ICD-9 diagnosis code of the recipient's primary 
a8 left justified diagnosis. If the dia gnosis code has less than five see DMS 


characters, the data must be left justified and fill the Diagnosis 
remaining positions to the right with spaces. Codes 


Optional 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 
INPAT-DX Alphanumeric, The ICD-9 diagnosis code of the recipient's primary 
left justified diagnosis. If the dia gnosis code has less than five see DMS 
characters, the data must be left justified and fill the Diagnosis 
remaining positions to the right with spaces. Codes 





Optional 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 
INPAT-FIRST-DT- Inpatient covered Period From and Thru Dates 


SVC " eoniate Alphanumeric The date that this billing period begins. 
First Date of Se ih (Translated from 


MMDDYY Julian calendar) 
format 


On or after 
01/01/2004 


Billing 


[Table C-16] 


INPAT-LAST-DT- Inpatient covered Period From and Thru Dates 
SVC 8 Alphanumeric The date that this billing period ends. On or before 


Last Date of UB-92 dates | (Translated from | |Table C-16] 01/31/2004 
Billing are In Julian calendar) 

MMDDYY 

format 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name and Length of Type of Data Valid Codes and Definitions Range of 
Description Data Valid Values 
[Number of 
Characters] 
and Format 








I NPAT-REVENUE- Missouri Medicaid currently utilizes a three byte 

CD 3 Numeric revenuecodes, while the UB-92 layout allows for a four 

Revenue Code This field byte revenue code. The three-digit code from 100 to 
should be 999 that represents the services that are billed on this 
right particular line item. The combined total number of 
justified, left accommodation and ancillary services billed cannot 
zero filled. exceed 28 lines per claim. 


100 - 999 





Accommodation revenue codes range from 10X 
through 21X. Ancillary revenue codes range from 22X 
through 99X. 


NOTE: Emergency Room (rev 450 and 459) and 


Ambulance (rev 540 to 549) may only be billed as 
inpatient if the patient is admitted to the hospital. 


[Table C-9] 


INPAT-UNITS-SVC 


Units of Service Numeric, Integer | The number of days per room rate for both covered 00001 - 99999 


Whole numbers and non-covered accommodations (revenue codes 100 
only are through 239). 

accepted for the [Table C-28] 

days. 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Drug Claim Validation, NCPDP Layout 


Field Name Length of Type of Data Valid Codes and Definitions Range of 
Data Valid Values 
[Number of 
Characters] 
and Format 


PHARM- Numeric — _ oo, 00000001 - 
PROCESSED- The Missouri Medicaid recipient identification number. | gqqq9999 
RECIP-!ID 
Recipient ID The Missouri Medicaid recipient identification number. 
Allowed characters are 0 through 9. 
NOTE: Reference the recipient's Medicaid card for the 
correct Medicaid identification number. 
[Table C-1] 
PHARM-FIRST-DT- Alphanumeric The dispense date. Multiple claims submitted via the 01/01/2004 - 
SVC 8 same transaction will have the same dispense date. 01/31/2004 
Dispensing Date 
NCPDP V3.C [Table C-16] 
dates are in 
CCYYMMDD 
format 


PHARM- Alphanumeric The prescription number of the prescription filled or 
PRESCRI PTION- 
NUM The only 
Pharmacy characters Default: None, this is a required field. 
Prescription allowed are A 
Number through Z, 0 [Table C-4] 

through 9, and 

hyphens. If less 

than 7 digits, left 









refilled. 


justify and fill the 


remaining 
positions to the 
right with 
Spaces. 
Performance Management Solutions Group 429 
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Field Name Length of Type of Data Valid Codes and Definitions 

Data 

[Number of 

Characters] 

and Format 
PHARM-DRUG- Numeric The metric or non-metric quantity of the drug being 
QTY-DSPNS dispensed. For example: A quantity of 100 would be 
Drug Quantity 0100. 
Dispensed 

[Table C-28] 


PHARM-DAYS- Numeric The estimated number of days the dispensed amount 
SUPPLY represents. A days supply greater than 365 Is invalid. 
Number of Days 
Supply [Table C-29] 
PHARM-DRUG- Numeric The code designated for the drug dispensed. The field 
NDC-CODE IS 5-4-2 format with no hyphens or spaces 
National Drug Default: None, this is a required field. 
Code 

[Table C-26] 





Performance Management Solutions Group 
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Range of 
Valid Values 


OOO001 - 99999 


001 - 365 


00000000000 - 
99999999999 
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Medical Claim Validation, CMS1500/ NSF Layout 


Field Name Length of Type of Data Valid Codes and Definitions Range of Valid 
Data Values 
[Number of 
Characters] 
and Format 


OUTPAT-CLAI M- 
TYPE 
Outpatient Claim 







Type 


Alpha This field indicates the claim type that has been 

Submitted by the Health Plan. Valid values are: 

M=Medical 

[Table C-97] 
OUTPAT- , _ ee 
PROCESSED- Numeric The Missouri Medicaid recipient identification 00000001 - 
RECIP-ID number. 99999999 
Recipient |D 

The Missouri Medicaid recipient identification 

number. Allowed characters are 0 through 9. 

NOTE: Reference the recipient's Medicaid card for 

the correct Medicaid identification number. 

[Table C-1] 


1 
OUTPAT-FIRST- This is the first date the service was performed. 
DT-SVC 8 This date cannot exceed the current date. 
First Date of Yyyy/mm/dd 
Service 


On or after 
01/01/2004 


Alphanumeric 


(Translated from 


Julian calendar) [Table C-16] 





OUTPAT-LAST-DT- This is the last date the service was performed. 
SVC 8 Alphanumeric This date cannot exceed the current date. On or before 
Last Date of Yyyy/mm/dd | (translated from 01/31/2004 
Service Julian calendar) [Table C-16] 
OUTPAT-PLACE- 2 Numeric 03 School See list of values 
OF-SVC 04 Homeless Shelter at left 
Place of 05 Indian Health Service Free-Standing Facility 
Service 06 Indian Health Service Provider-Based Facility 

Performance Management Solutions Group 43 | 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name Length of Type of Data Valid Codes and Definitions Range of Valid 
Data Values 


[Number of 
Characters] 
and Format 





07 Tribal 638 Free-Standing Facility 

08 Tribal 638 Provider-Based Facility 

11 Office 

12 Home 

13 Assisted Living Facility 

14 Group Home 

15 Mobile Unit 

20 Urgent Care Facility 

21 Inpatient Hospital 

22 Outpatient Hospital 

23 Emergency Room - Hospital 

24 Ambulatory Surgical Center 

25 Birthing Center 

26 Military Treatment Facility 

31 Skilled Nursing Facility 

32 Nursing Facility 

33 Custodial Care Facility 

34 Hospice 

41 Ambulance - Land 

42 Ambulance - Air or Water 

49 Independent Clinic 

50 Federally Qualified Health Center (FQHC) 
51 Inpatient Psychiatric Facility 

52 Psychiatric Facility - Partial Hospitalization 
53 Community Mental Health Center 

54 Intermediate Care Facility/Mentally Retarded 
55 Residence Substance Abuse Treatment Facility 
56 Psychiatric Residential Treatment Facility 
57 Non-Residential Substance Abuse Treatment 
Facility 

60 Mass Immunization Center 
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Field Name Length of Type of Data Valid Codes and Definitions Range of Valid 
Data Values 
[Number of 
Characters] 
and Format 


61 Comprehensive Inpatient Rehabilitation Facility 
62 Comprehensive Outpatient Rehabilitation 
Facility 

65 End Stage Renal Disease Treatment Facility 

71 State or Local Public Health Clinic 

72 Rural Health Clinic 

81 Independent Laboratory 

97 Parochial/Private Schools 

98 Schools 

99 Other Unlisted Facility 









[Table C-14] 
OUTPAT-UNITS- The number of units of services performed. Whole | OOO01 - 99999 
SVC numbers only are accepted for days or units. 
Units of Service 





[Table C-28] 
OUTPAT-DTL- Enter the HCPCS (CPT-4) procedure code that CPT-4; or HCPCS 
PROC represents the service performed. 2003 or 2004 
Procedure Code codes 


Required for Home Health and Outpatient claims. 
OUTFPATDH- 
PROECMOp-P 
‘eames = oe 
OUTFPATDH- 2 
PROCMOp-+— 
Preeedure 


Performance Management Solutions Group 
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[Table C-30] 
The 2 dia’ We , T , .; 
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EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


Field Name Length of Type of Data 
Data 


[Number of 
Characters] 
and Format 





OUTPAT-DX Alphanumeric, 


left justified 


——— 
OUTPAT-DX Alphanumeric, 
left justified 


OUTPAT-DX Alphanumeric, 


left justified 
Performance Management Solutions Group 
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Valid Codes and Definitions 





The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the diagnosis code has less 
than five characters, the data must be left justified 
and fill the remaining positions to the right with 
Spaces. 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


Required. 


[Table C-17] 

The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code has less 
than five characters, the data must be left justified 
and fill the remaining positions to the right with 
Spaces. 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


Optional 


[Table C-17] 

The ICD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code has less 
than five characters, the data must be left justified 
and fill the remaining positions to the right with 
Spaces. 





Range of Valid 
Values 





| tet o-fiteerrO 


See DMS 
Diagnosis Codes 


See DMS 
Diagnosis Codes 


See DMS 
Diagnosis Codes 
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Length of 
Data 


[Number of 
Characters] 
and Format 


Field Name 





Type of Data 


OUTPAT-DX Alphanumeric, 
left justified 
OUTPAT-DX Alphanumeric, 


- 
Performance Management Solutions Group 


a Division of Behavioral Health Concepts, Inc. 


2/16 Forum Blvd., Suite 4 
Columbia, MO 65203 
www. bhcinfo.com/PMSG 


T: 573.446.0405 
F: 573.446.1816 
E-mail: info@bhcinfo.com 


Valid Codes and Definitions 


Range of Valid 
Values 


Optional 






Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 

The ICD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code has less 
than five characters, the data must be left justified 
and fill the remaining positions to the right with 
Spaces. 


See DMS 
Diagnosis Codes 


Optional 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 

The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code has less 
than five characters, the data must be left justified 
and fill the remaining positions to the right with 
Spaces. 


See DMS 
Diagnosis Codes 


Optional 


Any decimal point needed in the diagnosis code is 
implied and must not be included. 


[Table C-17] 
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Outpatient Hospital Claim Validation, UB-92 Layout 
Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data Layout 
[Number of 
Characters] 
and Format 
OUTPAT-CLAI M- This field indicates the claim type that has 
TYPE been submitted by the Health Plan. Valid 
Outpatient Claim values are: 
Type 
O=Outpatient 


[Table C-97] 


OUTPAT- , | , — a 
PROCESSED- Numeric The Missouri Medicaid recipient 00000001 - 99999999 


RECIP-!ID identification number. 
Recipient |D 


The Missouri Medicaid recipient 
identification number. Allowed characters 
are 0 through 9. 


NOTE: Reference the recipient's Medicaid 

card for the correct Medicaid identification 

number. 

[Table C-1] 
OUTPAT-FIRST- This is the first date the service was 01/01/2004 - 
DT-SVC 8 Alphanumeric performed. This date cannot exceed the 03/31/2004 


UB-92 dates 
First Date of aa (Translated from current date. 


MMDDYY Julian calendar) 
format 


Service 
[Table C-16] 
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Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data Layout 

[Number of 
Characters] 
and Format 

OUTPAT-LAST-DT- This is the last date the service was 01/01/2004 - 

SVC 8 Alphanumeric performed. This date cannot exceed the 03/31/2004 

Last Date of UB-92 dates (Translated from | Current date. 

Service are 1n 


MMDDYY Julian calendar) 
format 
OUTPAT-UNITS- Numeric, Integer | The number of units of services performed. | 00001 -99999 
SVC Whole numbers only are accepted for days 
Units of Service or units. 






[Table C-16] 





[Table C-28] 
OUTPAT-DTL- Alphanumeric For outpatient hospital claims, a procedure | Blank; or CPT-4; or 
PROC code Is required only when the revenue HCPCS 2003 or 2004 
Procedure Code code range for outpatient services is 300 codes 

through 319. This revenue code range 

represents laboratory services. The 

appropriate CPT procedure code range for 

laboratory services is 80048 through 

89399. All other outpatient services must 

be designated by revenue code. (Note: 

Including a procedure code for services 

other than lab will not cause the encounter 

to reject as long as the revenue code is also 

present). 





[Table C-30] 
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Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data 
[Number of 
Characters] 
and Format 








OuUPPAF-DBHAL- 
PROCMOB—_+— 
= 


REVENUE CODE 3 
Revenue Code 


Numeric The three digit code from 100 to 999 
which represents the services that are billed | DMS to clarity whether 
on this particular line item. A revenue code | Or not less than 100 is 
is required on all Outpatient claims. For valid; 
those revenue codes representing lab 100 - 999 
services (300-319), a procedure code must 
also be submitted. 





[Table C-9] 
OUTPAT-DX Alphanumeric, The I|CD-9 diagnosis code of the recipient's 


3-5 left justified primary diagnosis. If the dia gnosis code See DMS Diagnosis 


has less than five characters, the data must | Codes 
be left justified and fill the remaining 
positions to the right with spaces. 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


Required. 
[Table C-17] 
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Field Name Length of 
Data 
[Number of 
Characters] 
and Format 


OUTPAT-DX Alphanumeric, 
3-5 left justified 

OUTPAT-DX Alphanumeric, 
3-5 left justified 
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Type of Data 


Valid Codes and Definitions 


The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code 
has less than five characters, the data must 
be left justified and fill the remaining 
positions to the right with spaces. 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


Optional 


[Table C-17] 

The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code 
has less than five characters, the data must 
be left justified and fill the remaining 
positions to the right with spaces. 


Optional 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


[Table C-17] 


Codes Found in File 


Layout 


See DMS Diagnosis 
Codes 


See DMS Diagnosis 
Codes 
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Field Name Length of 
Data 
[Number of 
Characters] 
and Format 


OUTPAT-DX Alphanumeric, 
3-5 left justified 

OUTPAT-DX Alphanumeric, 
3-5 left justified 
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Type of Data 


Valid Codes and Definitions 


The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code 
has less than five characters, the data must 
be left justified and fill the remaining 
positions to the right with spaces. 


Optional 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


[Table C-17] 

The |CD-9 diagnosis code of the recipient's 
primary diagnosis. If the dia gnosis code 
has less than five characters, the data must 
be left justified and fill the remaining 
positions to the right with spaces. 


Optional 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


[Table C-17] 


Codes Found in File 


Layout 


See DMS Diagnosis 
Codes 


See DMS Diagnosis 
Codes 
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Home Health Claim Validation, CMS1500/ NSF Layout 
Length of 
Data 


Field Name 





OUTPAT-CLAI M- 
TYPE 
Outpatient Claim 


Type 


OUTPAT- 
PROCESSED- 
RECIP-1D 
Recipient |D 


Q Alphanumeric 
Yyyyimm/dd | (translated 
from Julian 
calendar) 
OUTPAT-LAST-DT- 
SVC Q Alphanumeric 
Last Date of Yyyyimm/dd | (translated 
service from Julian 
calendar) 


Performance Management Solutions Group 


a Division of Behavioral Health Concepts, Inc. 


OUTPAT-FIRST- 
DT-SVC 

First Date of 
Service 





2/16 Forum Blvd., Suite 4 
Columbia, MO 65203 
www. bhcinfo.com/PMSG 


T: 573.446.0405 
F: 573.446.1816 
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Type of Data 
[Number of 
Characters] 
and Format 
Alphanumeric 


Numeric 


Valid Codes and Definitions 


This field indicates the claim type that has 
been submitted by the Health Plan. Valid 
values are: 


H=Home Health 


[Table C-97] 


The Missouri Medicaid recipient identification 
number. 


The Missouri Medicaid recipient identification 
number. Allowed characters are 0 through 9. 


NOTE: Reference the recipient's Medicaid 
card for the correct Medicaid identification 
number. 

[Table C-1] 

This is the first date the service was 
performed. This date cannot exceed the 
current date. 


[Table C-16] 


This is the last date the service was 
performed. This date cannot exceed the 
Current date. 


[Table C-16] 


Codes Found in File 
Layout 





8 characters, numbers 
only 





01/01/2004 - 
03/31/2004 


01/01/2004 - 
03/31/2004 
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Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data Layout 
[Number of 
Characters] 
and Format 


OUTPAT-UNITS- 
SVC 
Units of Service 





Numeric, 
Integer 


The number of units of services performed. OO001 - 99999 
Whole numbers only are accepted for days or 
units. 





[Table C-28] 


S. Alphanumeric Enter the HCPCS (CPT-4) procedure code that | 97003-99510 (not 
represents the service performed. consecutive) 
A4230 - Y2300 (not 
Required for Home Health and Outpatient consecutive) 
claims. 
[Table C-30] 


OUTPAT-DTL- 
PROC 
Procedure Code 





OUFPATB HAE 
PROCMCGD-P 
= 
MecHter Rt? } 
OuUFPATBH- 
PROC MOB+— 
Modttert{?} 


OUTPAT-DX 


- 
Alphanumeric, | The |CD-9 diagnosis code of the recipient's 
left justified primary diagnosis. If the dia gnosis code has see DMS Diagnosis 
less than five characters, the data must be Codes 


left justified and fill the remaining positions 
to the right with spaces. 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


on a Division of Behavioral Health Concepts, Inc. 
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Codes Found in File 
Layout 


Field Name Length of Type of Data Valid Codes and Definitions 


Data 


OUTPAT-DX 


[Number of 
Characters] 
and Format 





a ee 

[Table C-17] 

Alphanumeric, | The |CD-9 diagnosis code of the recipient's 

left justified primary diagnosis. If the dia gnosis code has 
less than five characters, the data must be Codes 


left justified and fill the remaining positions 
to the right with spaces. 


See DMS Diagnosis 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


Optional 


[Table C-17] 
Alphanumeric, | The |CD-9 diagnosis code of the recipient's 
left justified primary diagnosis. If the dia gnosis code has 


OUTPAT-DX . . 
See DMS Diagnosis 


less than five characters, the data must be Codes 
left justified and fill the remaining positions 
to the right with spaces. 


Optional 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 


[Table C-17] 
Alphanumeric, | The |CD-9 diagnosis code of the recipient's 
left justified primary diagnosis. If the dia gnosis code has 
less than five characters, the data must be Codes 
left justified and fill the remaining positions 
to the right with spaces. 


See DMS Diagnosis 





OUTPAT-DX 
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Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data Layout 
[Number of 
Characters] 
and Format 
Optional 
Any decimal point needed in the diagnosis 
code is implied and must not be included. 
[Table C-17] 


to the right with spaces. 


Optional 


Any decimal point needed in the diagnosis 
code Is implied and must not be included. 





OUTPAT-DX Alphanumeric, | The |CD-9 diagnosis code of the recipient's 
left justified primary diagnosis. If the dia gnosis code has | >€& DMS Diagnosis 
less than five characters, the data must be Codes 
left justified and fill the remaining positions 


[Table C-17] 
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Dental Claim Validation, CMS1500/ NSF Layout 
Field Name Length of Type of Data 
Data 


[Number of 
Characters] 
and Format 





OUTPAT-CLAI M- 
TYPE 
Outpatient Claim 


Type 


Alphanumeric 


- : 


8 Alphanumeric 
Yyyy/mm/dd (Translated from 
Julian calendar) 
OUTPAT-LAST-DT- . 
SVC 8 Alphanumeric 
Last Date of Yyyy/mm/dd (Translated from 
Service Julian calendar) 


Performance Management Solutions Group 


a Division of Behavioral Health Concepts, Inc. 


OUTPAT- 
PROCESSED- 
RECIP-1D 
Recipient |D 


OUTPAT-FIRST- 
DT-SVC 

First Date of 
Service 





2/16 Forum Blvd., Suite 4 
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T: 573.446.0405 
F: 573.446.1816 
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Valid Codes and Definitions 


This field indicates the claim type that has 
been submitted by the Health Plan. Valid 
values are: 


L=Dental 


[Table C-97] 


The Missouri Medicaid recipient 8 characters 


identification number. 


The Missouri Medicaid recipient 
identification number. Allowed characters 
are 0 through 9. 


NOTE: Reference the recipient's Medicaid 
card for the correct Medicaid identification 
number. 

[Table C-1] 

This is the first date the service was 
performed. This date cannot exceed the 
current date. 


01/01/2004 - 
03/31/2004 


[Table C-16] 

This is the last date the service was 
performed. This date cannot exceed the 
current date. 


01/01/2004 - 
03/31/2004 


[Table C-16] 


Codes Found in File 
Layout 
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Field Name Length of Type of Data Valid Codes and Definitions Codes Found in File 
Data Layout 
[Number of 
Characters] 
and Format 


OUTPAT-UNITS- 
SVC 
Units of Service 







The number of units of services performed. | OOO0O1 - 99999 
Whole numbers only are accepted for days 
or units. 


Numeric, Integer 





[Table C-28] 


OUTPAT-DTL- S. Alphanumeric Enter the HCPCS (CPT-4) procedure code DO120 - D9999 
PROC that represents the service performed. 
Procedure Code 

Required for Home Health and Outpatient 

claims. 

[Table C-30] 





OUFPATB HAE 
PROCMCGD-P 
Precedure 

MecHter Rt? } 
OuUFPATBH- 
PROC MOB+— 
Precedure 

Modttert{?} 
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PROCEDURE AND DIAGNOSIS CODE CRITERIA FOR CRITICAL FIELD VALIDATION 


Procedure Codes 


po MSEC CPCS 2003,2004 
| 0920-D9999 


F0100-E2599 (E2000-E2101) 

G0001-G9016 

H0001-H2037 (HO001-H2001) 
(J0120-J9999 


}0120-}9999 


KO001-K0620 


aS 

aaa 

aaa, 

SSO 

000-9900 
P| M0064-M0301 (MOOE4-MOBO2) 
| P2028-P9GTS 
po | Q0035-Q4077 (Q0034-Q9940) 
 ROOTOFROOPG 
pt (80009-59999) 
po T1000-T5999 (T1000-T2007) 
2020-53864 
SOOO 
SSS 
SS 





VALID PROCEDURE CODES 
PMSA Cs CPT-4 2004 Codees* 

00001-99501 0001F-0011F 
Pf 0001 T-0061T 
OOO OOOO ee aaa aaa 

* = does not account for deleted codes 
DMS Diagnosis Codes” 





V010-V8389 


X01-X02 
Y01-Y89 





' Source: Missouri Department of Social Services, Division of Medical Services, Payable Procedure Codes.xls 

° Thid., 

> Source: Missouri Department of Social Services, Division of Medical Services, Diagnosis Codes.xls 
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Appendix 6: Medical Record Review 
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DEC 9 
27 2004 
MISSOURI 
BOB HOLDEN DEPARTMENT OF SOCIAL SERVICES RELAY MISSOURI 
GOVERNOR DIVISION OF MEDICAL SERVICES for hearing and speech impaired 
P.O. BOX 6500 TEXT TELEPHONE 
1-800-735-2966 


JEFFERSON CITY 
65102-6500 


VOICE 
1-800-735-2466 


January 7, 2005 


Dear Medicaid Provider: 


The State of Missouri, Department of Social Services, Division of Medical 
Services (DSS/DMS) has contracted with Behavioral Health Concepts, Inc. (BHC) to 
perform the Missouri MC+ Managed Care Program Calendar Year (CY) 2004 
External Quality Review. This external quality review is being performed in 
accordance with the requirements of The Centers for Medicare and Medicaid 
Services (CMS) within 42 CFR 438.204(d). This federal regulation requires the 
performance of an annual, external independent review of the quality outcomes and 
timelines of, and access to, the services covered under each Managed Care 
Organization (MCO) contract. 


The State of Missouri, Division of Medical Services is a Health Oversight 
Agency as designated by the Health Insurance Portability and Accountability Act 
(HIPAA). BHC is a business associate of the State of Missouri. Specific references 
within the HIPAA “Standards for Privacy of individually Identifiable Health 
Information”, 45 CFR 164.512(d), provide that protected health information (PHI) 
may be disclosed without authorization in order to determine compliance with 
program standards. Accordingly, individual authorization from your patients for 
release of their medical records for this quality review is not required. 


We need very specific information to perform this review. We are also 
seeking to minimize your cost of complying with this requirement. We are 
requesting photocopies of certain portions of medical records for the patient names 
on the attached listing. Information is confidential and PHI. Please do not deliver 
these records via facsimile (fax) or electronic mail (email), follow the attached 
instructions. Records not received by the date will be considered undocumented 
encounters. , : 


The following are references that may be useful to you in complying with 
this request. The contract can be accessed through the Office of Administration, 
Division of Purchasing and Materials Management website at 


http://www. Oa. mo.gov/purch/webimaging/Homepage.htm. You can ee under 


*“AN EQUAL OPPORTUNITY/AFFIRMATIVE ACTION EMPLOYER** 
services provided on a nondiscriminatory basis 


Page 2 


the referenced contract number. Documents regarding these final rules can be 


found at the CMS website at http://www.cms.hhs.gov/medicaid/managedcare. 


State of Missouri/BHC Contract # C301154001 

Medicaid Managed Care Final Rule 42 CFR Part 400, et.al. June 14, 2002) 
External Quality Review Final Rule 42 CFR Parts 433 and 438 January 24, 2003) 
HIPAA Privacy Final Rule 45 CFR Parts 160 and 164 (August 14, 2002) 


Thank you for your cooperation in the CY 2004 annual External Quality 
Review. If you have any questions about the submission of the medical records, 
please contact Ms. Mona Prater, BHC, Inc. at 573-446-0405. 





Director, Program Management 


SKL/em 
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MEDICAL RECORD SUBMISSION 
INSTRUCTIONS 


Missouri MC+ 
January 7, 2008 


Dear Clinic Manager: 





Enclosed is a listing of medical records and green face sheets with provider and patient demographic 
information for individual patients selected from your office for the 2004 MC+ Managed Care External 
Quality Review. As explained in the enclosed cover letter from Sandra Levels, Director of Program 
Management, Division of Medical Services, the purpose of this review is to examine encounter claims 
that were submitted for patients enrolled in the MC+ Managed Care Program (Medicaid) during 
calendar year 2004. 


We are interested in all services provided to these patients by the designated provider from January 
1, 2004 through March 31, 2004. This information is used to document the volume and type of 
services provided to MC+ Managed Care Program Members and to validate the accuracy and 
completeness of the State encounter claims database. 


For each medical record that is requested, please provide the following: 

e Face/Demographic sheet enclosed or other documentation that identifies the patient receiving 
services and primary care provider, January 1, 2004 through March 31, 2004. This information 
includes: 

Patient Name 

Medicaid ID Number 

Date of Birth 

Provider Name 

Provider Number 

e Documentation of all services (professional, physician’s/doctor's orders, 
laboratory test results) from January 1, 2004 through March 31, 2004. 
Sources for this information may include: 

Primary Diagnosis 
Progress Notes 
Laboratory findings 
Treatment Plans 
Claim Forms or Superbills 
O Flow Sheets 
Behavioral Health Providers: 
Due to the sensitive nature of these records, please include only the primary diagnosis 
code, claim information, and minimum necessary information to support the diagnosis and 
procedure codes for which services were billed during the specified time frame (January 1, 
2004 — March 31, 2004). Please do not send raw test data, protocols, or shadow charts. 
We also recommend that you de-identify names of individuals related to the patient in 
progress notes. 
Include any and all information to support the procedures for which a claim was 
submitted. 
Records not received by the due date will be considered undocumented encounters 
(see back) 


Oo O00 0 0 


OOO: 0: .O 
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lf more than one record is requested, please separate patient records using the 
matching green face sheets provided, and complete the summary sheet. If you do 
not have a record for the patient, please indicate this on the summary sheet and 
return to BHC, Inc. by the due date. 


Please note that BHC will not reimburse providers or copy services for copy and 
postage costs. Please do not submit invoices. 


In the interest of confidentiality, please DO NOT FAX or E-MAIL any forms or 
portions of medical records. 


Mail to: 


Mariya Chumak, BS, Research Associate 
Behavioral Health Concepts, Inc. 
2800 Forum Bivd., Ste. 3A 
Columbia, MO 65203 


We want to express our appreciation in advance for your participation in this 
process and sincerely hope to minimize any inconvenience and interruption in 
providing clinical services to your patient 
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Adolescent Well Care Abstraction Tool 


Patient Name Pt ttt ttt | tt ft 


Last 
First 
m m d d y y y y 


Date of Birth P| | | | tf ft 


Missing = 99999999 


Provider Name Pt tt tte te tt 


Last 
First 
Name of MCO LJ} Community Care Plus (1) LJ Family Health Partners (5) 
(Check only one) LJ Mercy Health Plan (2) LL] FirstGuard (6) 
LJ HealthCare USA (3) L} Blue Advantage Plus (7) 


LJ Missouri Care (4) 


Abstractor 
Initials 

Date of 
abstraction 
Data entry 
operator initials 
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Search the medical record for a well care visit during the calendar year 2003 


Source of 
Documentation: LJ Medical Record (1) 
LJ Claim Form (2) 
LJ} Both (3) 
L} None (0) 
Documented Health and Developmental History 
Components L} Yes (1) 
of Well Care L} No (0) 
Visit: 
(Check all that Physical Exam 
apply) LJ Yes (1) 
LJ} No (0) 
Anticipatory Guidance 
L} Yes (1) 
L} No (0) 


Date of Well Care 
Visit m m d d y y y y 


Unless ALL components 
above are checked, code 


Missing = 99999999 


Acceptable Procedure Codes: 


Missing = 99999 99393 - 99395 
Insufficient Information = 22222 
Don't Know = 88888 





See list to the right of Procedure Codes. Does procedure code match one of these? Acceptable Diagnosis Codes: 
Procedure Code 
Match U Yes (1) V70.5 V70.9 
LI No (0) , V70.6 
V70.8 





Diagnosis Code 
Decimal is implied. 


Start at left. If only 3 or 
4 digits, leave the right 
spaces blank. 

Missing = 99999 


Insufficient Information 
= 22222 


Don't Know = 88888 


Diagnosis Code UO Yes (1) 
Match LI No (0) 
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Adolescent Immunization Abstraction Tool 


Patient Name 


Date of Birth: 
Missing = 99999999 


Provider Name 


First 
Name of MCO L} Community Care Plus (1) LJ Family Health Partners (5) 
(Check only one) LJ Mercy Health Plan (2) LJ FirstGuard (6) 
LJ HealthCare USA (3) LJ Blue Advantage Plus (7) 
L} Missouri Care (4) 


Abstractor Initials 
Date of abstraction 


Data entry operator 
Initials 


Start Time 
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Search the medical record for the complete immunization history from 1991 through 2003 





Source of Documentation: LJ} Medical Record (1) 
Check One LJ Claim Form (2) 
LJ Both (3) 
L} None (0) 
Type of Documentation LJ} Dated Immunization History (1) 
Check One L} Immunization Certificate (2) 
LJ Both (3) 
LJ None (0) 
Is There Evidence of a History of: 
Measles LJ Yes (1) 
LJ} No (0) 
Mumps LJ Yes (1) 
L} No (0) 
Rubella LJ Yes (1) 
LJ} No (0) 
Measles Seropositive Test Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
Mumps Seropositive Test Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
Rubella Seropositive Test Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
MMR Date 1 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
MMR Date 2 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
MMR Date 3 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
MMR Date 4 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
m m d d 
First Birthday | | {| {| 1991 
m m d d 
m m d d 
Were two MMRs completed after the member's first (1991) AND before the member's L} Yes (1) 
fourth birthday (1994)? LJ No (0) 
Was one of the MMRs completed after the member's fourth birthday (1996) and before. LI) Yes (1) 
the member's thirteenth birthday (2003)? LJ} No (0) 
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Source of Documentation: LJ} Medical Record (1) 
(Check all that apply) LJ Claim Form (2) 

LJ Both (3) 

L} None (0) 
Type of Documentation: LJ} Dated Immunization History (1) 
(Check only one) L} Immunization Certificate (2) 

LJ Both (3) 

L} None (0) 
Is there documented evidence of a history of Hep B? LJ Yes (1) 

L} No (0) 

Hep B Seropositive Test Result Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
Hep B Date 1 m m d d y y y y 


Missing = 99999999 
Not Applicable = 88888888 


At delivery/birth = 11111111 


Hep B Date 2 m m d d y y y y 

Missing = 99999999 

Not Applicable = 88888888 

Hep B Date 3 m m d d y y y y 

Missing = 99999999 

Not Applicable = 88888888 

Hep B Two Dose Regimen Date 1 m m d d y y y y 

Missing = 99999999 

Not Applicable = 88888888 

Hep B Two Dose Regimen Date 2 m m d d y y y y 

Missing = 99999999 

Not Applicable = 88888888 

Were three Hep Bs completed by the members’ 13th birthday? LI Yes (1) 
LJ} No (0) 

Was one dose of the two-dose regimen and 2 other doses of Hep B completed by the L} Yes (1) 

members’ 13th birthday? L} No (0) 

h h m m 
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Record ID 555 Page 1 


Medical Record Abstraction Tool 


Patient Name Jane Doe 

Date of Birth 11/4/1968 

Patient DCN OOOOO0001 

Provider Name Sample Provider Name 

Provider Address Some Street Address Some Suite Number 


First Date of Service 2/14/2004 


Abstractor Initials | | [| 


Data entry operator initials | | | 


Start Time | tt | 


Examine only the information provided in physician and professional documentation. DO NOT 
use the CMS-1500, any claim forms, or any claim histories. 


Medical Record 


Element Comparison 


2/14/2004 


Date of Service 


Decimal is implied. Start at left. If onl 


Missing = 99999 
Comment (Required if Error Type = Other) 


UTERINE LEIOMYOMA NOS 


Primary Diagnosis 





Comment (Add description from medical record; Required if Error Type = Other) 





Error Analysis Codes: 1 = Data Entry; 2 = Insufficient Information; 3 = Incorrect (No match); 8 = No Error (Match); 
9 = Missing; 0 = Other 
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Record ID: 555 Page 2 
Procedure Code To be coded by reviewer 


Decimal is implied. Start at left. If only 3 or 4 digits, leave the right spaces blank. 


Not Enough Information = 22222 | tt 


Comment (Required if Error Type = Other) 





To be coded by reviewer 


Procedure Description 
Comment (Add description from medical record; Required if Error Type = Other) 


Referrals Documented in the Medical Record (check all that apply; only if not related to 
the claim validated) 








None (0) 
Laboratory (1) 














Pharmacy (2) 
Specialist (3) 
Radiology (4) 
Other (5) 


See next page for the procedure code and procedure code description to be validated. 
Does the medical record documentation adequately support the procedure code and 
description? 
Yes No 

If no, Reason (check only one): 


Not enough information (e.g., the date of service and information are 
present, but there is not enough information to make a determination) (1) 


Upcoded (2) 
Incorrect (3) 
Missing (9) 

Other (4) 








































































































Comment 


Error Analysis Codes: 1 = Data Entry; 2 = Insufficient Information; 3 = Incorrect (No match); 8 = No Error (Match); 
9 = Missing; 0 = Other 


BHC © 2004 Behavioral Health Concepts, Inc. 


Record ID: 555 Page 3 


Examine the CMS-1500 or any claim forms. If there is no claim form or history, code 
as missing. 





Claim Form or History 


Element shearne rison 


re fm {mfololy [it], fe Be. 
miaeooen | | 11111111 





Comment necreauied if Error Type = Other) 


Primary Diagnosis “Xe 


Decimal is implied. Start at left. If only 3 or 4 digits, leave the right spaces blank. 
Missing = 99999 | ft fb 


Comment (Required if Error Type = Other) 











UTERINE LEIOMYOMA NOS 


Primary Diagnosis 


Comment (Required if Error Type = Other) 


Procedure Code 
Comment (Required if Error Type = Other) 





ANESTHESIA FOR INTRAPERITONEAL 
PROCEDURES IN LOWER ABDOMEN NOT 
OTHERWISE SPECIFIED 


Procedure Descripti 
Comment (Required if Error Type = Other) 


Error Analysis Codes: 1 = Data Entry; 2 = Insufficient Information; 3 = Incorrect (No match); 8 = No Error (Match); 
9 = Missing; 0 = Other 
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Appendix 7: MC+ MCO Compliance with Managed Care 
Regulations 
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EXTERNAL QUALITY REVIEW OF MiSssSoURI MC+ MANAGED CARE PROGRAM: REPORT OF FINDINGS, 2004 


2004 
Missouri MC+ 
FeV at= ke [stem @r- a= om mole la-lan 





External Quality Review 


SITE VISIT AGENDA 
Morning 
TIME ACTIVITY ATTENDEES LOCATION 








8:30 — 9:00 INTRODUCTION - BHC, Inc. 
OPENING Dr. Janet Reed Anca Geana 
Mona Prater Sachi Rath 
Myrna Bruning 
MCO Attendees 
9:00 — 10:00 Validation of Performance BHC, Inc. — 
Improvement Projects Dr. Janet Reed 
MCO Attendees 
9:00 — 10:00 Validation of Performance BHC, Inc. — 
Measures — Document Review | Anca Geana 
Sachi Rath 
Will need access to computers 
and individuals who can answer | MCO Attendees 
questions and have access to 
the data files that produce the 
HEDIS measures. 
Computer access may be 
needed throughout the 
day. 
9:00 — 10:00 Compliance — Document BHC, Inc. — 
Review Mona Prater 
Myrna Bruning 
0:00-0:15 | Reviewer Break—Orderlunch | SS~=~“C~SsS~—S—SSSSSC 
10:15 — 11:45 Validation of Performance BHC, Inc. — 
Measures -- Interviews Dr. Janet Reed 
Anca Geana 
Sachi Rath 
lO:15 — 11:45 
(continued) MCO Attendees 
10:15 — 11:00 Compliance Review — BHC, Inc. — 
Interview With Mental Health Mona Prater 
Myrna Bruning 
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11:00 — 11:45 Compliance Review — BHC, Inc — 


Interview with Provider Mona Prater 
Relations/Provider Myrna Bruning 
Services/Contract Management 

MCO Attendees 





Afternoon 
TIME ACTIVITY ATTENDEES LOCATION 








11:45 — 1:00 | Working Lunch on Site BHC Staff 
Reviewer Meeting 
1:00 — 2:30 Validation of Performance BHC, Inc. — 
Measures — Interviews Dr. Janet Reed 
(continued) Anca Geana 
Sachi Rath 
MCO Attendees 
1:00 — 1:45 Compliance Review — BHC, Inc. — 
Interviews - Member Services/ Mona Prater 
Utilization/Case Management Myrna Bruning 
MCO Attendees 
1:45 — 2:30 Compliance Review — BHC, Inc. — 
Interview - Quality Assurance Mona Prater 
Myrna Bruning 
MCO Attendees 
2:30 — 3:15 Compliance Review — BHC, Inc — 
Interview — Organization Dr. Janet Read 
Leadership Mona Prater 
Myrna Bruning 
MCO Attendees 
2:30 — 3:15 Validation of Performance Anca Geana 
Measures — Continued Sachi Rath 
Document Review/Interview 
follow-up MCO Attendees, as needed 
3:15 —400 | Exit Interview Preparation | SiteReviewTeam SSSS~sSC“CS*~‘“~*~S 
4:00 — 4:30 Exit Conference BHC, Inc. — 
Dr. Janet Reed Mona Prater 
Anca Geana Myrna Bruning 
Sachi Rath 
MCO Attendees 
Performance Management Solutions Group 465 
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Behavioral Health Concepts, Inc. 


February |, 2005 


RE: SITE VISIT AGENDA AND DOCUMENT REVIEW 


We are in the process of finalizing plans for the on-site reviews of each MCO. We are 
providing the following information in an effort to make preparation for the on-site 
review as efficient as possible for you and your staff. The following is information or 
persons needed at the time of the on-site review at (MCO name). 


Performance Improvement Projects 


One hour is scheduled in the morning to conduct follow-up questions, review databases, 
and provide verbal feedback to the MCO regarding the planning, implementation, and 
credibility of findings from the Performance Improvement Projects (PIPs). Any staff 
responsible for planning, conducting, and interpreting the findings of PIPs should be 
present during this time. The review will be limited to the projects and findings 
submitted at the end of 2004. Please be prepared to review databases and any data 
collection forms not originally submitted. 


Performance Measure Validation 


As you know, BHC is in the process of validating the following three performance 
measures: 


=™ HEDIS 2004 Use of Appropriate Medication for Persons with Asthma 
® HEDIS 2004 Adolescent Immunization Status, Combination #1 
™ HEDIS 2004 Adolescent Well Care Visits 


BHC is following the CMS protocol for validating performance measures. The goals for 
this process are to: 


Evaluate the accuracy of Medicaid performance measures reported by the MCO; and 
Determine the extent to which Medicaid-specific performance measures calculated 
by the MCO followed specifications established by the Division of Medical Services. 
These specifications consist of the HEDIS 2004 Technical Specifications. 
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To complete this process we will review the following documents while on-site: 


=" Data Integration and Processes Used to Calculate and Report 
Performance Measures 


|. Documentation of the performance measure generating process 

2. Report production logs and run controls 

3. Documentation of computer queries, programming logic, or source code (if 
available) used to create denominators, numerators and interim data files - for each 
of the three measures 

4. Code mapping documentation 

5. Documentation of results of statistical tests and any corrections with justification for 
such changes, if applicable - for each of the three measures 

6. Documentation showing confidence intervals of calculations when sampling 
methodology used — for each of the three measures 

7. Description of the software specifications or programming languages instructions 
used to query each database to identify the denominator, and/or software manual 

8. Source code for identifying the eligible population and continuous enrollment 
calculation — for each of the three measures 

9. Description of the software specification or programming languages used to identify 
the numerator 

10. Programming logic and/or source code for arithmetic calculation of each measure to 
ensure adequate matching and linkage among different types of data 


= Sampling Validation 


|. Description of software used to execute sampling sort of population files 

2. Source code for how samples for hybrid measures were calculated 

3. Policies to maintain files from which the samples are drawn in order to keep 
population intact in the event that a sample must be re-drawn or replacements made 

4. Documentation that the computer source code or logic matches the specifications 

set forth for each performance measure, including sample size and exclusion 

methodology 

Documentation of “frozen” or archived files from which the samples were drawn 

6. Documentation assuring that sampling methodology treats all measures 
independently, and there is no correlation between drawn samples 


i 


Performance Measure Interviews 


In addition to the documentation reviews, interviews will be conducted with the 

person(s) responsible for: 

™ Overseeing the process of identifying eligible members from MCO data sources for 
the measures to be validated; 

= Programming the extraction of required elements from the MCO data sources for 
the measures to be validated; 

™ Integrity checks and processes of verifying the accuracy of data elements for the 
measures to be validated; 

= Overseeing the process of medical record abstraction, training, and data collection 
for the measures to be validated; and 


= Contractor oversight and management of any of the above activities. 
On-site activities may also include, but are not limited to, the following: 


= Demonstration of HEDIS software 
™ Demonstration of the process for extracting data from MCO databases 
= Possible data runs for identifying numerator and denominator cases 


Compliance Review 


The final activity to prepare for during the on-site visit will be the compliance review. 
Many documents will be reviewed prior to the on-site visit. Interviews will also occur 
with staff from the Division of Medical Services before the on-site review to enable 
BHC to use the time at the MCO as efficiently as possible. The following information 
will be needed on the date of the on-site review: 


Compliance Documents 


Member Handbook 

Provider Handbook 

Provider Agreements 
(requested) Protocols or Policies 
Marketing Plan 


Compliance Interviews 


The agenda sent previously contained the times for interviews on the day of the site 
visit. We have tried to group questions together to make this process more effective. 
If you can get the following groups together for the interview process it would be very 
helpful. 


Quality Assurance Director and staff 
Mental Health 
Provider Services/Provider Relations/Contract Management 
Member Services/Utilization Management/Case Management staff 
Organization Leadership — 

Plan Director 

Medical Director 

Quality Assurance Director 


There are concurrent activities and interviews occurring during both the morning and 
afternoon. If separate conference rooms or meeting spaces can be arranged, this will 
make the process much easier to coordinate. Also, the on-site review team will need to 
order a working lunch on the day of the visit. If you can provide a name and telephone 
number on the day of the review of a service in the vicinity to accommodate this, we 
will appreciate it. 


The MCO staff involved in any of the referenced interviews or activities, or anyone 
identified by the MCO, are welcome to attend the introduction or the exit interview. 


Your assistance in organizing the documents, individuals to be interviewed, and the day’s 


activities is appreciated. If you have questions, or need additional information please let 
me know. 


Sincerely, 


Mona Prater 
Assistant Project Director 


Cc: Dr. Janet Reed, Project Director 
Plan Administrator 


Judy Muck, Division of Medical Services 


Attachment: On-Site Review Agenda 
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BHC MCO COMPLIANCE REVIEW SCORING FORM 

This document is used to score the number of items met for each regulation by the MCO. 
|. Review all available documents prior to the site visit. 
2. Follow-up on incomplete items during the site visit. 


3. Use this form and the findings of Interviews and all completed protocols to complete the Documentation and Reporting Tool and rate 
the extent to which each regulation is met, partially met, or not met. 


Scores from this form will be used to compare document compliance across all MCOs. 

0 = Not Met: Compliance with federal regulations could not be validated. 

| = Partially Met: MCO practice or documentation indicating compliance was observed, but total compliance could not be validated. 

2 = Met: Documentation is complete, and on-site review produced evidence that MCO practice met the standard of compliance with federal 


regulations. 


Atha fare | 
O = Not Met 
Site Visit and 1 = Partially Met 
Contract Compliance Tool Federal Regulation Description Comments | a akeliate 2 = Met 


Subpart C: Enrollee Rights and Protections 





Zz. 
2. 438.100(a) Enrollee Rights: General Rule 
2. 
2. 438.10(b) Enrollee Rights: Basic Rule 
Alternative Language: Prevalent 
2.15.2(e), 2.8.2 438.10(c)(3) Languages 
Language and format: 
2.8.2, 2.8.3, 2.6.2(n)(2) 438.10(c)(4,5) Interpreter Services 
Information Requirements: 
2.6.1(a)1, 2.6.2(n)1 438.10(d)(1) (i) Alternative Formats 
2.6.1(a)1, 2.6.2(n)2 - dot 
point 35, 2.6.2(q), 2.8.2, Information Requirements: 
2.8.3 438.10(d)(1)(ii)and (2) Easily Understood 
2.3.5, 2.6.1(a)2/3, 2.6.2(k)1, 
2.6.2(n), 2.6.2(n)(2), Enrollee Rights: Information, 
2.6.2(q) 438.10(f) Free Choice 
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Site Visit and 


Atha fare| 
O = Not Met 
1 = Partially Met 





Federal Regulation 


Contract Compliance Tool 


8 2.6.2(n)(2) 438.10 (g) 
2.4, 2.4.5, 2.4.5(a)2-4, 
9 2.20.1(all), 3.5.3(f) 438.10(i) 
2.2.6(a), 2.2.6(b), 
10 2.6.1(a)(3), 2.6.2(j), 2.9.1 438.100(b)(2) (iii) 
2.6.2(j), 2.30.1, 2.30.2, 
11 = 2.30.3 438.100(b)(2)(iv,v) 
12 | 2.6.2(j), 2.4.8, 2.13, 2.14 438.100(b)(3) 


13 2.2.6, 2.14.3, 2.14.8, 2.14.9 438.100(d) 


Total Enrollee Rights and Protections | —$<—— |—_—— 


Description Comments a laketiare k3 
Information to Enrollees: 

Physician Incentive Plans 

Liability for Payment and Cost 

Sharing 

Specific Enrollee Rights: 

Provider-Enrollee 

Communications 

Right to Services, including right 


of refusal. Advance Directives 


Right to Services 
Compliance with Other State 
Requirements 


Subpart D: Quality Assessment and Performance Improvement 


Subpart D: Quality Assessment and Performance Improvement: Access Standards 


14 438.206(b)(1)(i-v) 


15 | 2.7.1(e), 2.7.1(f), 2.14.8 438.206(b)(2) 


16) 2.13 438.206(b)(3) 
2.3.2, 2.3.18, 2.7.1(bb), 
We 263 2524 2 aS 438.206(b)(4) 


18 2.4, 2.20.1(d) 
2.3.14(a)2, 2.14.1, 2.14.4(a- 
UOe |) 2.27 S58 


438.206(b)(5) 


438.206(c)(1)(i-vi) 


20 2.2.6(a)1-3, 2.17.1 438.206(c)(2) 
2.3.2, 2.6.2(k) 1-4, 2.14.11, 
2 a2 Scone 438.208(b) 


22  2.6.2(m), 2.14.11, 2.5.3(e) 
2.12.10, 2.14.2(c), 2.14.11, 
2.17.5, Attachment 3 - 
Children with Special 

23 Healthcare Needs 


438.208(c)(1) 


438.208(c)(2) 


Performance Management Solutions Group 


a Division of Behavioral Health Concepts, Inc. 


T: 573.446.0405 
F: 573.446.1816 
E-mail: info@bhcinfo.com 


2/16 Forum Blvd., Suite 4 
Columbia, MO 65203 
www. bhcinfo.com/PMSG 


Availability of Services: Provider 
Network 

Access to Well Woman Care: 
Direct Access 


Second Opinions 

Out of Network Services: 
Adequate and Timely Coverage 
Out of Network Providers: Cost 
Sharing 


Timely Access 

Cultural Considerations 
Primary Care and Coordination 
of Healthcare Services 


Care Coordination: | dentification 


Care Coordination: Assessment 


| 2 = Met 
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Atha fare| 
O = Not Met 
Site Visit and 1 = Partially Met 
Contract Compliance Tool Federal Regulation Description | Comments a akeliate | 2 = Met 
Care Coordination: Treatment 
24 2.7.1, 2.12, 2.14.11 438.208(c)(3) Plans 
2.3.8, 2.3.7, 2.6.1(k)(3), 
25 | 2.14.6, 2.14.7 438.208(c)(4) Access to Specialists 
2.2.1(i), 2.3.7, 2.7.4, 2.9.2, 
2.10.2, 2.14.1, 2.14.2(a-h), 
26 | 2.14.2(d)1-2 438.210(b) Authorization of Services 
27 2.15.4, 2.14.2(d)6 438.210(c) Notice of Adverse Action 
2.6.2(k)(3), 2.14.2(d)6, 
28 2.15.4(a-c), 2.16.3(e) 438.210(d) Timeframe for Decisions 
Compensation for Utilization 
29 2.17.5(b) 438.210(e) Management Decisions 
2.4.8, 2.7.1, 2.7.1(y), Emergency and Pos-stabilization 
30) -.2.7.3(v), 2.14.2 438.114 pgs 24/25 Rev. Checklist 
Subpart D: Quality Assessment and Performance Improvement: Structure and Operation Standards 
General Rules for Credentialing 
31 = 2.17.2(n), 2.17.5(c), 2.30.2 438.214(a,b) and Recredentialing 
Nondiscrimination and Provider 
32 2.2.6(b)(c) 438.214(c) and 438.12 Discrimination Prohibited 
33) «2.31.5 438.214(d) Excluded Providers 
Other State Requirements: 
34 | 2.3.9, 2.3.17 438.214(e) Provider Selection 
2.6.2(n)(2), 2.6.2(s) (all), 438.226 and Disenrollment: Requirements 
35 2.6.2(u) 438.56(b)(1-3) and Limitations 
2.o. 2 Due, 225.0). 2.0. (00); Disenrollment Requested by 
36 «=62.6.2® 438.56(c) Enrollee 
Procedures for Disenrollment -- 
37 = -2.6.2(r,s-1,t) 438.56(d) Pgs 29/30 Rev. Checklist 
Timeframe for Disenrollment 
38 = 2.6.2(u) 438.56(e) Determinations 
39 =2.15, 2.15.3(a,b) 438.228 Grievance Systems 
2.6.1(a)(18), 2.16.2(c), 
2.31.2(a)8, 2.31.3, 3.5.1, Subcontractual Relationships 
AQ 3,512). 3-915 438.230(a,b) and Delegation 
Subpart D: Quality Assessment and Performance I mprovement: Measurement and Improvement 
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_| Federal Regulation 





_| Contract Compliance Tool 


41 2.17.2(d) 438.236(b)(1-4) 
42 2.17.2(d) 438.236(c) 
43 2.17.2(d,f) — 438.236(d) 


44 | 2.17.1, 2.17.5 438.240(a)(1) 


438.240(b)(1) and 


45 2.17.5(d) 438.240(d) 
438.240(b)(2)(c) and 
46 2.17, 2.17.3, Attachment 6 438.204(c) 


47  2.17.5(b) 438.240(b)(3) 
48 2.17.5 

Attachment 6 - State Quality 
49 Strategy 


438.240(b)(4) 


438.240(e) 


50 2.25 
2.25(all) - 2.25.1, 
51 | 2.25.2(a,b), 2.25.3, 2.25.4 


438.242(a) 
438.242(b)(1,2) 


52 | 2.26.1, 2.29.1 438.242(b)(3) 





53 2.15 438.402(a) 


54 2.15.2, 2.15.5(a), 2.15.6(a) 438.402(b)(1) 


55 2.15.6(a) 438.402(b)(2) 
2.15.2(a), 2.15.5(a), 
56 2.15.6(a,b) 438.402(b)(3) 


57 2.15.2(e), 2.15.4(a),2.6.2(q) 438.404(a) 
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_| Description 


Baciiite 

| 0 = Not Met 

Site Visitand | 

Comments lake tiate k3 | 2 = Met 

| There is very little 
in the contract 
compliance tool 

regarding practice 


guidelines. 


Adoption of Practice Guidelines 
Dissemination of Practice 
Guidelines 

Application of Practice Guidelines 
-- Pgs 32/33 of Rev. Checklist 
Quality Assessment and 
Improvement Program 

Basic Elements of MCO QI and 
PIPs 


Performance Measurement 
Basic elements of MCO QI and 
PIPs: Monitoring Utilization 
Basic elements of MCO QI and 
PIPs 


Program Review by State 
Health Information Systems 
Basic Elements of HIS 


Basic Elements of HIS 


Total Quality improvement and Assessment 


Subpart F: Grievance Systems 
Grievance and Appeals: General 
Requirements 
Grievance and Appeals: Filing 
Authority 


Grievance and Appeals: Timing 
Grievance and Appeals: 
Procedures 

Notice of Action: Language and 


_ Format 


47\ 


= Partially Met 





EXTERNAL QUALITY REVIEW OF MissouRI MC+ MANAGED CARE PROGRAM: REPORT OF FINDI NGS, 2004 


| Rating 


O = Not Met 
Site Visit and 1 = Partially Met 








Contract Compliance Tool _| Federal Regulation Description | Comments | a laketiare k3 | 2 = Met 
58 2.15.4(b) 438.404(b) Notice of Action: Content 
59 2.15.4(c) 438.404(c) Notice of Action: Timing 
Handling of Grievances and 
60 2.15.5(b,c,d), 2.15.6(h,i,j) 438.406(a) Appeals: General Requirements 
2.15.6(g) 2.15.6(h) 2.15.6(i) Handling of Grievances and 
61 2.15.6(j) 438.406(b) Appeals: Special Requirements 


Resolution and notification: 
Grievances and Appeals - Basic 
62 2.15.5(e), 2.15.6(k) 438.408(a) rule 
Resolution and notification: 
Grievances and Appeals - 
63 2.15.5(e,f), 2.15.6(k-I) 438.408(b,c) Timeframes and extensions 
Resolution and notification: 
Grievances and Appeals - 
64 2.15.5(e), 2.15.6(k,m) 438.408(d)(e) Format and content 
Resolution and notification: 
Grievances and Appeals - 
Requirements for State fair 
65 2.15.2(i), 2.15.6(m) 438.408(f) hearing 


66 2.15.6(n,o) 438.410 Expedited resolution of appeals 
Information about the grievance 
systems of providers and 

67 2.15.2(c), 3.5.3(c) 438.414 Subcontractors 


68 2.15.3 438.416 Recordkeeping and reporting 
Continuation of Benefits while 
the MCO/PIHP Appeal and the 

69 2.15.6(p) 4388.420 State Fair Hearing are Pending 


70 | 2.15(a,r) 438.424 Effectuation of reversed appeals 
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